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ABSTRACT 

Modern aircrew training devices (ATDs) are equipped 
with sophisticated hardware and software capabilities, known as 
advanced instructional features (AIFs), that permit a simulator 
instructor to prepare briefings, manage training, vary task 
difficulty/fidelity, monitor performance, and provide feedback for 
flight simulation training missions. The utility and use of the AIF 
capabilities of Air Force ATDs were explored by means of a survey of 
534 simulator instructors from Air Training Command, Military Airlift 
Command, Strategic Air Command, and Tactical Air Command training 
sites. The primary purpose of the survey was to provide a database 
that could be used in defining the requirements for ATD procurement 
and in developing future ATD training programs. The results indicated 
that, in general, the features that were rated highest were those 
used for training management, variation of task difficulty/fidelity, 
and monitoring student performance. The level of AIF use was affected 
somewhat by hardware and/or software unreliability, implementation 
time, functional limitations, and design deficiencies. However, the 
presumed training value of an AIF was the most important determinant 
of its use. Recommendations were^made concerning the AIF capabilities 
of future ATDs and research aimed at determining the principles of 
effective AIF use. The survey instruments are appended. 
( Author /KC) 
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SUMMARY 



Modern Aircrew Training Devices (ATDs) are equipped with 
sophisticated hardware and software capabilities, known as Advanced 
Instructional Features (AIFs), that permit a simulator instructor (SI) to 
prepare briefings, manage training, vary task difficulty/fidelity, monitor 
performance, and provide feedback for flight simulation training 
missions. The utility and utilization of the AIF capabilities of Air 
Force ATDs was explored by means of a survey of 534 Sis from Air Training 
Command, Military Airlift Command, Strategic Air Command, cind Tactical Air 
Command training sites. The primary purpose of the survey was to provide 
a database that could be used in defining the requirements for ATU 
procurements and in developing future ATD training programs. In general, 
the features that were rated highest were those used for training 
management, variation of task difficulty/fidelity, and monitoring student 
performance. The level of AIF use was affected somewhat by hardware 
and/or software unreliability, implementation time, functional 
limitations, and design deficiencies. However, the presumed training 
value of an AIF was the most important determinant of its use. 
Recommendations are made concerning the AIF capabilities of future ATUs 
and research aimed at determining the principles of effective AIF use. 
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PREFACE 
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Human Resources Laboratory Technical Planning Objective 3, the thrust of 
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thrust is to identify and demonstrate cost-effective simulator training 
strategies and training equipment capabilities for use in developing and 
cost-effectively maintaining the combat readiness of Air Force aircrew 
members. More specifically, the research was conducted under the Air 
Combat Training, Research subthrust, the goal of which is to provide a 
technology base for training high-level and quicKly perishable skills in 
simulated combat environments. Dr. Gary S. Thomas (AFHRL/OTU) was the 
Project Monitor; Dr. Donald J. Polzella and Dr. David C. Hubbard were the 
Co-Investigators, both of whom were employed by the University of Dayton 
Research Institute. 
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AIRCREW TRAINING DEVICES: UTILITY AND UTILIZATION 
OF ADVANCED INSTRUCTIONAL FEATURES 
(PHASE IV - SUMMARY REPORT) 



I. INTRODUCTION 

An Aircrew Training Device (ATD) is a ground-based substitute 
aircraft that permits student flight crews to fly in a safe and carefully 
controlled environment. It is frequently assumed that the training value 
(i.e., utility) of an ATD is a function of its fidelity or capability for 
simulation. According to Adams (1972), this assumption could be 
unwarranted. 

I would not consider the money being spent on flight simulators 
as staggering if we knew much about their training value, which 
we do not. We build flight simulators as realistic as 
possible, . . . but the approach is also a cover-up for our 
ignorance about transfer because in our doubts we have made 
costly devices as realistic as we can in the hopes of gaining 
as much transfer as we can. [In the past], the users have been 
willing to pay the price, but the result has been an avoidance 
of the more challenging question of how the transfer might be 
accomplished in other ways, or whether all that complexity 
(i.e., fidelity) is really necessary, 'pp. 616-617) 

It is important to realize that an ATD is primarily an instructional 
device that is designed to facilitate the acquisition of flight crew 
skills. Thus, the training value of an ATD is determined not by the 
degree to which it faithfully simulates a particular aircraft but by the 
way that it is used (Caro, 1973). Yet, it appears that military ATDs are 
more often thought of as substitute aircraft than as instructional tools. 
A recent report by the United States General Accounting Office (1983) 
concluded that the armed services have not sufficiently analyzed their 
training requirements for simulators. Nor have they adequately 
incorporated simulators into their training programs. In justifying the 
purchase of ATDs, the services have focused instead on "duplicating the 
actual weapon systems and their surroundings . . . with little reference 
to how the devices could meet training needs" (p. 4). The GAO report 
makes two recommendations to the Secretary of Defense: 

1. Approve budget requests for flight simulators only after the 
services have analyzed their training needs and proven that the 
needs cannot be met with existing simulators. 

2. Require the services to incorporate simulators into their 
training programs, (p. 7). 

It is clear from the report that if the armed services are to follow these 
recommendations, they must better utilize and understand the instructional 
capabilities of ATDs. 
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Advanced Instructional Features 

In order to fulfill its function as an instructional device, an ATD 
is equipped with sophisticated hardware and software capabilities that 
permit a Simulator Instructor (SI) to brief, control, monitor, and provide 
feedback during simulated training missions* These capabilities, some of 
which are listed in Table 1, are known as Advanced Instructional Features 
(AIFs)* The list was compiled from several sources, but it was drawn 
primarily from Semple, Cotton, and Sullivan's (1981) extensive report 
describing the AIF capabilities of various military and commercial 
devices. Instructional features are expensive to implement, especially 
those features that require the development of complex software. In order 
to justify these costs, some questions concerning the present and 
potential utility and utilization of AIFs should be answered: How 
frequently and easily are AIFs used? Are Sis adequately trained to use 
AIFs? Do AIFs have significant training value? 

Table 1. Advanced Instructional Features 



BRIEFING FEATURES 

* Recorded Briefing permits SI to provide student with information about 

the simulator and/or a training mission through audiovisual media 
presentation. 

* Detnonstration permits SI to demonstrate optimal aircrew performance 

by means of prerecording and subsequently playing back segments of 
simulated flight. 

* Instructor Tutorial provides ''I with self-paced programined instruction 

in the capabilities and uso of the simulator. 

TRAINING MANAGEMENT FEATURES 

* Total System Freeze permits SI to suspend simulated flic^'t by freezing 

all system parameters . 

* Reset permits SI to return the simulated aircraft to a stored set of 

conditions and parameters. 

* Crash and/or Kill Override permits SI to allow simulated flight to 

continue without Interruption following a **crash" or "kill." 

* Automated Adaptive Training is the computer-controlled variation in task 

difficulty, complexity, and/or sequence based on student's 
performance. 

* Programmed Mission Scenarios are computer-controlled training missions 

based on pre-programmed event sequences. 

* Manual Mission Control permits SI to modify programmed scenarios during 

a training session. 
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Table 1 ( Continued ) 
VARIATION OF TASK DIFFICULTY/FIDELITY FEATURES 

* Automated Malfunction Insertion permits SI to preprogram a sequence 

of aircraft component malfunctions and/or emergency conditions. 

Manual tjalfunctioh Insertion permits SI to modify preprogrammed 
malfunctions during , a training session. 

* Environmental permits SI to vary environmental conditions such as wind 

direction and velocity^ turbulence, temperature, and visibility. 

Dynamics, permits SI to vary flight dynamics characteristics such as 
stability, system gain, cross-coupling, etc. 

* Motion permits SI to provide student with platform motion system cues 

such as roll, pitch, lateral, and vertical. 

* Partial Freeze permits SI to freeze any one or a combination of flight 

parameters. Variations of this feature include: 

Flight System Freeze , which permits SI to simultaneously freeze 
flight control and propulsion systems, position, altitude, and 
heading; 

Position Freeze , which permits SI to simultaneously freeze latitude 
and longitude; and 

Attitude Freeze , which permits SI to simultaneously freeze pitch, 
bank, and heading. 

MONITORING FEATURES 

Closed Circuit TV permits SI to monitor student's behavior from the 
instructor console. 

Repeaters/Annunciators provide SI with replicas or analog 
representations of flight instruments and controls at the 
instructor console. 

* Instructor Console Displays permit SI to monitor parameters and 

procedures at the Instructor console by means of alphanumeric 
and/or graphic cathode-ray tube (CRT) displays of performance data 
(in the survey this was referred to as Parameters and Procedures 
Monitoring). 

Automated Performance Alert provides SI with visual and/or auditory 
signals that indicate specific performance deficiencies. 
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Table 1 (Concluded) 

FEEDBACK FEATURES 

* Record/Playback permits SI to store and subsequently play back a 

' segment of simulated flight, 

* Automated Performance Feedback provides student with visual and/or 

auditory signals (Including verbal messages) that identify 
performance deficiencies. 

Automated Voice Controller is the computer-based technology that 
simulates the role of controller by combining speech generation, 
speech recognition, and situation awareness capabilities. ; 

* Hard Copy provides an alphanumeric and/or graphic record of performance 

data. 

* Performance Scoring provides a metric(s) that summarizes aircrew task 

performance during a simulated mission. 



These features were surveyed during one or more phases of the project. 



Answers to these questions have not been fully provided, but relevant 
information is available^ One source of information is a series of 
reports describing the automated instructional system on the Advanced 
Simulator for Pilot Training (ASPT) at Williams AFB, Arizona (Faconti & 
Epps, 1975; Faconti, Mortimer, & Simpson, 1970; Fuller, Waag, & Martin, 
1980; Knoop, 1973). The ASPT is a sophisticated research device that 
incorporates advanced visual and motion systems, A-10 and F-16 cockpits, 
extensive AIF capability, and an automated performance measurement 
system. Notwithstanding the apparent training potential of the ASPT, 
Gray, Chun, Warner, and Eubanks (1981) found that Sis tended to use the 
device in a fairly conventional manner. With few exceptions, the 
instructional features were rarely used. 

AIF utility information is available in an important series of 
reports by R. G. Hughes et al. (Bailey & Hughes, 1980; Bailey, Hughes, & 
Jones, 1980; Hughes, 1979; Hughes, Hannon, & Jones, 1979; Hughes, Lintern, 
Wightman, Brooks, & Singleton, 1982). The reports provide conceptual 
models for AIF-based simulator training programs and present experimental 
evidence aimed at determining the training value of particular features. 
It is clear from these reports that effective AIF-based simulator training 
is practicable, but systematic analyses of AIF utility and utilization 
patterns are required before optimal training programs, of the kind 
envisioned by Knoop (1973), can be designed: 

The software which will comprise flight simulators of the future 
will consist primarily of sophisticated advanced training programs 
which automatically step the student through training, measure his 
performance at each step, diagnose his problems, and alter the 
difficulty of various tasks which are commensurate with his skill, 
(p- 583) 



A great deal of information concerning AIFs can be found in the 
design guides of Caro, Pohlinan, and Isley (1979) and Easter, Kryway, 
Olson, Peters, Slemon, and Obermayer (1986a, 1986b) and in the Semple et 
al. (1981) report mentioned previously. This latter report is probably 
the most comprehensive source. It was based on interviews of ATU 
personnel at 12 Air Force, Navy, Army, Coast Guard, and commercial 
training .sites and is one of seven reports comprising the Air Force 
Simulator Training Requirements and Effectiveness Study (STRES). The 
report describes over 20 AIFs and discusses each in terms of its 
operation, related instructional features, instructional value, observed 
applications, utilization information, related research, and design 
considerations. The interviews were "guided" by a checklist of topics, 
but they were not highly structured. This approach afforded the 
investigators flexibility in, exploring particular topics, but it precluded 
systematic analyses of the data. 

The Present Investigation 

The present investigation was conducted at the request of the 
Simulator System Program Office (SimSPO) of the Air Force Systems Command, 
Aeronautical Systems Division (AFSC/ASD). The objectives of the 
investigation were: 

1. To document and compare the utilization (i.e., frequency and ease 
of use) of AIFs, 

2. To document and compare the utility (i.e., training value) of 

AIFs, 

3. To document and compare instructor training in the use of AIFs, 

4. To compare the utility and utilization patterns of AIFs in 
replacement (e.g., basic, primary, lead-in, initial, formal, transition) 
and continuation (e.g., advanced, follow-on, refresher, operational) 
training units, 

5. To compare the utility and utilization patterns of AIFs across 
the Air Force Major Commands (MAJCOHs), and 

6. To make recommendations concerning the inclusion of AIFs on 
future ATDs and their use in current and future training programs. 

Tne approach of the investigation consisted of surveying Sis from the 
MAJCOMs in three phases between March ly82 and September 1984. Instructor 
pilots (IPs) and weapon director instructors from Tactical Air Command 
(TAC) were surveyed in Phase I. IPs, flight engineers, and 
radar/navigators from Air Training Command (ATC), Military Airlift Command 
(MAC), and Strategic Air Command (SAC) were surveyed in Phase II. 
Electronic warfare instructors, aarial gunnery instructors, and weapon 
system officers from ATC, SAC, and TAC were surveyed in Phase III. The 
results of those phases are documented in three previous reports 
(Polzella, 1983, 1985; Polzella & Hubbard, 1985). The present report 
completes, the final phase of the investigation with a summary of the 
entire database. 
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The purpose of this sumnary is threefold: (a) to extract findings 
that are .general izable across MAJCOMs and/or levels of training, (b) to 
determine differences that may exist across MAJCOHs and/or levels of 
training, and (c) to make recommendations for the inclusion of AIFs on new 
ATDs;^ The information contained in this report should be supplemented 
with experimental data, since the conclusions and recommendations are 
derived from a database consisting almost entirely of subjective judgments. 



The subjects were 534 simulator-qualified flight crew instructors 
(Sis) assigned to various ATC (T-50, T,t51, T-5), MAC (C-5A, C-141, CH-3, 
HH-53, C-130), SAC (FB-lllA, T-4, B-^52 weapon system trainer [WST]), and 
TAC (F-4E, F-4G, F-15, A-10, E-3A) training sites. The subjects included 
IPs, weapon directors (WDIs), flight engineers (IFEs), radar/navigators 
(IRNs), aerial gunners (lAGs), and electronic warfare officers (EWIs), 
The distribution of particular Sis among the various levels of training 
and ATD sites that were surveyed in the various phases are shown in 
Appendix Tables- A-l through At3. Also included are the Sis' mean (and 
standard deviation) number of hours of instructor experience, A summary 
of this information is presented in Table 2. 

Questionnaires 

The questionnaires that were used to survey the instructors are shown 
in Appendix B, Phases I, II, and III. It was necessary to construct a 
separate version of the questionnaire for each phase of the investigation, 
due to differences in the training requirements and capabilities of the 
various ATD sites. However, the resulting versions were similar in most 
respects. 

The first page of each questionnaire requested information concerning 
instructor experience, a description of a typical simulator training 
session, and general comments and/or recomrnendations. The second page of 
each questionnaire listed the features that were to be surveyed, along 
with their definitions. The list consisted of from 14 to 17 AIFs 
(depending on the version) drawn from Table 1. A total of 19 AIFs were 
surveyed during one or more phases of the project. 

Table 2 . Simulator Instructors (Sis) Surveyed During This Investigation 



II. METHOD 



Subjects 



Command 



ATD 



Number of Sis 



Mean number of 
instructor hours 



ATC 



T-50 
T-51 

T-5 (EW) 



29 
21 
19 



173.6 
129.4 
287.4 
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Table 2 (Concluded) 
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TAC 


F-4E/G 


26 
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F-4G (EW) 


32 


95.6 




F-15 


39 


144.0 




A-10 


38 


21.7 




A-10 (EW) 


33 


75.0 




E-3A FS 


8 


351.3 




E-3A HS 


23 


448.1 



On the remaining pages of the survey were five questions concerning 
the utility and utilization of each feature. For Phases I and II, the 
questions were worded as follows: 

1. How often have you used each instructional feature? 

2. How easy is it to use each instructional feature? 

3. How much training did you receive in the use of each 
instructional feature? 

4. Rate the training value of each instructional feature. 

5. Rate the potential training value of each instructional feature. 

The questions were altered somewhat for Phase III, at the request of 
TAC/DOT: 

1. During five typical missions, how often did you use each 
instructional feature? 

2. How difficult/easy is it to use each instructional feature? 

3. How inadequate/adequate was the training you received in the use 
of each instructional feature? 

4. As presently implemented on your system, how useful is each 
instructional feature? 

5. Based on the definitions alone and not your experience, how 
potentially useful is each instructional fealure? 

Except for Question 3, which asked Sis to rate the adequacy rather 
than the amount of training they received, the Phase III questionnaire was 
comparable to that used for Phases I and II. 
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For the fifth question, Sis were to assume that they had no prior 
knowledge of the features and to base their responses on the feature 
definitions alone. This question was included in order to achieve a 
common basis for comparison a'nong all Sis. This was not otherwise 
possible because the various ATDs were not similarly equipped. 

Responses to each question were indicated by checking the appropriate 
interval along a 7-point, successive-category rating scale. (On certain 
questions a 0-interval was included for indicating "not applicable.") The 
intervals of each scale were labeled with descriptive adjectives, such as 
slightly useful, fairly useful, moderately useful, extremely useful, and 
indispensable, in order to facilitate responding and aid in interpretation 
of the ratings. Additional space was provided for comments. 

Procedure 

The questionnaire was administered on-site to various sized (N = 5 to 
10) groups of Sis. The Sis were briefed on the purpose of the 
investigation and copies of the questionnaire were distributed and 
thoroughly reviewed prior to being filled out. The questionnaire could be 
ccmpleted in approximately 30 minutes. 



III. RESULTS 

The Sis' responses to each question \/ere coded as 0 (not applicable) 
to 7 (the maximum possible rating). The ratings were classified by ATD 
(e.g., F-4, F-15, etc.), type of training (e.g., replacement, 
continuation), and AIF (e.g., recorded briefing, demonstration, etc.). 
The resulting data matrix was unbalanced due to differences in the number 
of Sis and in the AIF capabilities of the various ATD sites. In most 
cases, this required that the data from each ATD be analyzed separately. 
The results of these analyses are reported elsewhere (Polzella, 1983, 
1985; Polzella & Hubbard, 1985). A summary of the results follows. 

General Trends 

Interrelations Among the Variables 

At every training site, there were clear interrelations among the 
ratings, as indicated by the intercorrelations between the ratings of each 
feature across the five questions. The observed Pearson correlation 
coefficients' ranged from -.05 to +.80; 95% of the coefficients were 
positive and significant. Thus, the following generalization describes 
reasonably well the data obtained at every site: A feature's rating on any 



' The Pearson correlation coefficient is a statistical measure of 
association that can range from -1 to 1. A value of 1 represents a 
perfect positive relationship; and a -1, a perfect negative or inverse 
relationship. A value of zero indicates no linear relation. 
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question can be predicted with greater-than-chance accuracy, given its 
rating on any other question. For example, the more useful a feature was, 
the more frequently.it was used, the easier it was to use, the greater and 
more adequate was the training received in its use, and the greater was 
its potential training value. 

Multiple linear regressions were used to determine the degree to 
which the frequency of AIF use could be predicted from the remaining 
utility and utilization ratings. Three potential predictors were 
evaluated: the ease of AIF use; the amount (or adequacy) of training 
received; and AIF training value, a composite variable representing the 
average of the ratings on Questions 4 (training value) and 5 (potential 
training value). 

A separate analysis was computed for each ATU site, and the results 
are summarized in Table 3. Ease of use, training received, and training 
value, together, accounted for roughly 40% of the variability in the 
frequency-of-use ratings. The standardized regression coefficients 
(betas) associated with each variable are indications of the strength of 
that variable as a predictor; i.e., the larger the beta, the greater the 
predictability. It can be seen, for virtually every ATD, that the most 
important predictor of a feature's use was its training value. For TAC 
nbn-EW ATDs, ease oF use, but not amount of training received, was also an 
important predictor. For ATC, MAC, and SAC non-EW ATUs, both ease of use 
and amount of training received were moderately important predictors, 
whereas for EW ATDs, neither ease of use nor adequacy of training received 
tended to be important. 

Overview of the Data 

An overview of the data can be obtained by examining Figures 1 
through 6, in which unweighted mean ratings of the frequency of use, ease 
of use, training value, potential training value, amount of training 
received, and adequacy of training received are shown for each AIF that 
was surveyed. The unweighted means were calculated by averaging over the 
means obtained at each site. Although each unweighted mean is an unbiased 
average, unaffected by differences in the number of Sis that were 
surveyed, it is only representative of ATU sites at which the particular 
AIF-capability was present and/or was surveyed. 

The features are grouped according to function in these figures. 
Briefing features are those used for briefing the student and/or SI prior 
to or during a training mission. Training management features permit the 
SI to control the structure and sequencing of tasks within a training 
mission. Variation of task difficulty/fidelity features permits the SI to 
control the difficulty of simulated missions through variations in ATD 
fidelity, configuration, or task load demands. Monitoring features permit 
the SI to monitor parameters (i.e., aircraft states) and procedures (i.e., 
discrete actions performed by the student in accordance with prescribed 
standards) at the instructor console in the form of alphanumeric and/or 
graphic cathode-ray tube (CKT) displays of performance data. Finally, 
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Table 3* Multiple' Linear Regression Analyses of Frequency of AIF-use 
on Lase of AIF-use, Training Received by Sis, and AIF Training Value 

ATC MAC * SAC TaC 



T-50 T-51 T-5 C-5/ C-130 CH-3/ FB-lllA FB-UIA T-4 B-52 WST F-4E/ F-4G F-15 A-lO A-lO E-3A E-3A 

(EW) C-141 HH-53 (EW) (EW) (EW) F-4G (EW) (EW) FS MS 

Multiple R .74 .71 .55 .53 .70 .70 .65 .66 .53 .53 .66 .68 .65 .65 .50 .72 .52 
Ease of Use 

Beta .28 .17 .16 .19 .22 .29 .15 .11 .03 -.06 .37 -.08 .29 .29 .02 .33 .32 

p .00 .00 .12 .00 .00 .00 .00 .02 .67 .52 .00 .25 .00 .00 .82 .UO .00 

Training Received by SI^ 

Beta .18 .16 -.02 .25 .19 .18 .24 .35 -.13 .20 .13 03 .05 .26 .06 -.03 .12 

p .00 .00 .83 .00 .00 .00 .00 .00 .02 .03 .05 .71 .15 .00 .50 .78 .18 

AIF Training Value^ 

Beta .46 .52 .49 .30 .46 .40 .42 .35 .57 .45 .37 .68 .43 .26 .46 .54 .35 

p .00 .00 .UO .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 

^ Refers to "amount of training received" (non-electronic warfare) or "adequacy of training received" (electronic warfare). 

b Refers to the average of "training value" and "potential training value" (ron-electronic warfare) or tne average of 
"usefulness" and "potential usefulness" (electronic warfare). 
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Figure 1. Frequency of AIF Use. Mean ratings of each AIF for frequency of use. 
Circles represent RTU means: squares. CTU means. Numbers in 
parentheses indicate the number of ATD sites where the respective 
AIF was rated. 
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Figure 2. Ease of AIF Use. Mean ratings of each AIF for ease of use. Circles 
represent RTU means: squares. CTU means. Numbers in parentheses 
indicate the number of ATD sites wtiere the respective AIF was rated. 
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Figure 3. Training Value of AIF. Mean ratings of each AIF for trammg value. 
Circles represent RTU means: squares. CTU means. Numbers in 
parentheses indicate the number of ATD sites where the respective 
AIF was rated. 
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Figure 4. Potential Training Value for AIF. Mean ratings of each AIF tor potential 
training value. Circles represent RTU means: squares. CTU means. 
Numbers in parentheses indicate the number of ATD sites where the 
respective AIF was rated. 
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Figure 5. Amount of Training Received in AIF Use. Mean ratings of each AIF 
tor the amount of training received. Circles represent RTU means: 
squares* CTU means. Numbers in parentheses indicate the number of 
ATD sites where (he respective AIF was rated. 
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Figure 6. Adequacy of Training Received in AiF Use. Mean ratings of each 
AIF for adequacy of training received. Circles indicate the number 
of ATD sites where the respective AiF was rated. 
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feedback features permit the SI to provide the student with various forms 
of performance feedback. 

Two general trends are revealed in the figures. First, training 
management, variation of task difficulty/fidelity, and monitoring features 
tended to receive the highest ratings, whereas briefing and feedback 
features tenf<ed to receive the lowest ratings. This trend was observed at 
every ATD si*.a. 

The second general trend revealed in the figures is that there were 
differences in the ratings of features across the two levels of training. 
The figures indicate that certain features tended to receive higher 
ratings from replacement training unit (RTU) Sis (e.g., performance 
scoring), whereas others tended to receive higher ratings from 
continuation training unit (CTU) Sis (e.g., programmed mission 
scenarios). However, there were many exceptions to this trend, which are 
noted in the sections that follow. 

Utility and Utilization Ratings of Each AIF 

A more detailed summary of the data is presented in the tables that 
follow. The tables list means and standard deviations of the Sis' ratings 
of the frequency of use, ease of use, amount of training received, 
adequacy of training received, training value, and potential training 
value for each of the 19 AIFs that were surveyed, respectively. The data 
are tabulated according to MAJCOH, and statistics are listed separately 
for the two levels of training and for every ATD having the particular AIF 
capability (assuming that it was included on the given questionnaire). 
ATC ATDs include T-50, T-51, and T-5 (electronic warfare, EW), RTU only- 
MAC ATDs include C-5/C-141, C-130, and CH-3/HH-53, RTU and CTU. SAC ATDs 
include FB-lllA, FB-lllA (EW), T-4 (EW), and 6-52 WST (EW), RTU and CTU. 
TAC ATDs include F-4E/F-4G, F-4G (EW), F-15, A-10, A-10 (EW), E-3A Flight 
Simulator, and E-3A Mission Simulator, RTU and CTU. Note that the data 
from some of the training sites were combined (e.g., C-5/C-141, 
CH-3/HH-53, and F-4E/F-4G). It seemed appropriate to combine these data 
since the respective training missions were highly similar and the 
comparable mean ratings were nearly identical. 

Briefing Features 

Recorded Briefing (Table 4) 

Incidence . Recorded briefing was surveyed at every ATD site; it was 
available for use on only four devices: T-5, T-4, A-10, and E-3A mission 
simulator. 

Utility and utilization . Except for T-5 Sis, who regularly used 
recorded briefing, the majority of instructors never used the feature; and 
several were not even aware that the capability existed. Instructors' 
written comments suggested a preference for informal briefings, which 
could be adapted to the particular needs of individual students and 
instructors. 
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Table 4. Recorded Briefing 



Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 

of Use-R 
Ease 

of Use-C 



ATC 



MAC 



SAC 



TAC 



T-50 



T-61 T-5 
(EW) 



C-5/ C-130 
C-141 



CH-3/ 
HH-53 



FB-111A FB-111A T-4 B-52 WST 
(EU) (EW) (EW) 



4.2 
(2.1) 



5.8 
(1.2) 



1.2 
(0.4) 

2.5 

(1.0) 

4.4 

(1.7) 

4.8 
(1.3) 



F-4E/ 
F-4G 



F-4G 
(EW) 



F-15 A-10 



A-10 
(EM) 



E-3A 
FS 



1.2 
(0.8) 

1.0 
(0.0) 

5.0 
(0.0) 



E-3A 
MS 



1.5 
(0.9) 

1.8 
(1.3) 

4.6 
(1.5) 

3.5 
(2.1) 



00 



Amount of 

Training 

Received-R 



1.2 
(0.8) 



1.3 
(0.8) 



Amount of 

Training 

Received-C 



1.3 
(1.2) 



2.2 
(1.8) 



Adequacy of 
Training 
Recei ved-R 



6.3 
(1.0) 



4.4 
(2.4) 



Adequacy of 

Training 

Received-C 

Training 
Value-R 

Training 
Value-C 



5.6 
(1.4) 



6.5 
(1.1) 

2.1 
(1.2) 

6.0 
(0.8) 



5.2 
(1.5) 

2.5 
(0.7) 



3.6 
(2.1) 

3.3 
(0.6) 
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Potential 

Training 

Value-R 

Potential 
Training 
Value-C 



4.7 
(2.0) 



5.0 4.9 
(1.5) (1.8) 



3.9 3.7 3.6 
(1.6) (1.5) (1.8) 



4.5 4.5 
(1.7) (1.4) 



3.0 
(1.8) 



3.3 
(1.8) 



3.3 
(1.5) 



3.9 3.0 3.8 3.3 3.5 3.9 3.7 4.1 5.0 4.6 

(1.8) (1.1) (1.6) (1.5) (2.0) (1.4) (1.6) (1.8) (1.0) (1.9) 



4.2 4.9 4.2 3.6 2.9 2.7 3.3 2.9 1.5 4.4 

(1.7) (1.4) (1.8) (1.4) (1.7) (1.4) (1.3) (1.7) (0.7) (1.5) 



Comparisons across MAJCOMs and levels of training . The potential 
utility of recorded briefing tended to be highest for ATC ATDs and lowest 
for TAC ATDs. For six of the seven TAC ATDs (all except the F-4E/F-4G), 
the RTU Sis* potential training value ratings were higher than those of 
the:CTU Sis. (This difference was significant for the E-3A flight 
simulator.) In contrast, for three of the four SAC ATDs (all except the 
FB-IHAK the CTU Sis' potential training value ratings were higher than 
those of the RTU Sis. 

Demonstration (table 5) 

Incidence . Demonstration was surveyed at every ATD site. It was 
available for use on^ six devices: C-130, CH-3/HH-53, FB-lllA, A-10, A-10 
(EW), and E-3A mission simulator. 

Utility and utilization . The ratings of the demonstration feature 
were among the lowest ratings given for any feature. There were two major 
complaints at each site: First, implementing the feature was 
time-consuming and often unreliable. Second, an enormous effort was 
required to update and maintain current scenarios through software 
development, which resulted in an insufficient number of demos to meet 
training requirements. 

Comparisons across WJCOHs and levels of training . There were no 
significant differences In utility and utilization across MAJCOMs. The 
E-3A flight simulator RTU Sis rated demonstration higher in potential 
training value than did the CTU Sis; however, there were no other 
significant differences across the two levels of training. 

Instructor Tutorial (Table 6) 

Incidence. Instructor tutorial was surveyed at every ATU site; 
however, none of the ATDs included this capability. This feature differed 
from the other AIFs surveyed in that its purpose was the instruction of 
Sis in the operation of the ATD. 

Utility and utilization . The potential training value ratings were 
all in the moderate range of the scale. The instructors* written consents 
suggested that they prefer "hands-on" experience and/or "face-to-face" 
tutorials on the operation of the instructor's console. 

Comparison across MAJCOMs and levels of training . The potential 
training ratings were statistically equivalent across the MAJCOMs and 
between the two levels of training. 

Training Management Features 

Total System Freeze and Reset (Tables 7 and 8) 

Incidence * Total system freeze and reset were surveyed at every ATU 
site. Total system freeze was availaole for use on every device except 
the F-4E/F-4G, F-15, and E-3A flight simulator (but it was present on the 
E-3A mission simulator). Reset was available on every device except 
the E-3A mission simulator. 
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Tables. Demonstration 



Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC MAC SAC TAC 



T-50 T-51 T-5 C-5/ C-130 CH-3/ FB-lllA FB-lllA T-4 B-52 WST F-4E/ F-4G F-15 A-10 A-10 E-3A E-3A 









(EW) 


C-141- 




HH-53 




(EW) 


(EW) 


(EW) 


F-4G 


(EW) 






(EW) 


FS 


MS 


Frequency 
of Use-R 






- 




1 .6 
(1.0) 


1.5 
(1.5) 


1.7 
(0.8) 














1 .8 
(1.3) 


1.1 

(0.0) 




3.3 
(2.3) 


Frequency 
of Use-C 










2.4 
(1.6) 


1.6 
(1 .3) 


1.4 
(0.7) 














1.2 
(0.4) 


1.1 

(0.3i 




3.0 
(Z.O) 


Ease 

of Use-R 










4.1 
(1.1) 


4.5 
(2.1) 


2.0 
(1.0) 














3.6 
(1.8) 


2.2 
(1.0) 




4.8 

(1.5) 


Ease 
of Use-C 










4.5 
(1.1) 


3.8 
(1.9) 


2.6 
(1.2) 














4.5 
(0.7) 


3.0 
(1.6) 




4.0 
(1.4) 


Amount of 

Training 

Received-R 










2.4 
(1.1) 


1.5 
(1.2) 


2.3 
(1.5) 














2.2 
(1.4) 






3.0 
(1.9) 


Amount of 

Training 

Received-C 










3.2 
(1 .2) 


1.8 
(1.2) 


2.0 
(1.4) 














1.3 
(0.9) 






3.7 
( 2. 1 ) 


Adequacy of 

Training 

Received-R 






























2.2 
(1.2) 






Adequacy of 

Training 

Received-C 






























3.5 
(1.8) 






Training 
Value-R 










3.4 
(1.5) 


.■?.o 

(2.3) 


3.3 
(0.6) 














3.8 
(1.7) 


1.7 
(0.6) 




5.3 
(0.9) 


Training 
Value-C 










3.4 
(1.8) 


2.7 
(1.7) 


3.9 
(1.9) 














3.3 
(1.2) 


2.2 
(1.3) 




5.0 
(0.0) 


Potential 

Training 

Value-R 


4.6 
(1-6) 


5.5 
(1-2) 


5.0 


4.4 
(1.5) 


4.1 
(1.3) 


4.5 
(2.0) 


3.8 
(1.9) 


5.5 
(1.5) 


4.6 
(1.7) 


5.2 
(1.3) 


4.1 
(1.7) 


4.4 
(1.8) 


4.5 
(1.5) 


4.5 
(1.6) 


3.7 
(2.1) 


5.2 
(0.8) 


5.1 
(1.8) 


Potential 
Training 
Value-C 








4.6 
(1.6) 


4.7 
(1.5) 


4.3 
(1.9) 


4.2 

(1.5), 


4.7 
(1.6) 


5.9 
(1.2) 


4.5 
(1.9) 


4.5 
(1.6) 


4.6 
(1.7) 


3.6 
(1.8) 


4.5 
(1.6) 


3.6 
(1.6) 


1.5 

(0.7) 


5.6 
(1.1) 



Table 6> Instructor Tutorial 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 

of Use-R 
Ease 

of Use-C 

Amount of 

Training 

Received-R 

Amount of 

Training 

Recelved-C 

Adequacy of 

Training 

Received-R 

Adequacy of 

Training 

Received-C 

Training 
Value-R 

Training 
Value-C 



Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



SAC 



T-50 



T-51 T-5 
(EW) 



C-5/ 
C-141 



C-130 



CH-.3/ 
HH-53 



FB-lllA FB-lllA T-4 B-52 WST 
(EW) (EW) (EW) 



TAC 



F-.4E/ 
F-.4G 



F-4G 
(EW) 



F-15 A-10 



A-10 
(EW) 



E-3A 
FS 



E-3A 
MS 



Potential 

Training 

Value-R 

Potential 

Training 

Value-C 



4.0 4.4 4.2 
(1.6) (1.5) (1.9) 



4.3 3.8 
(1.5) (2.2) 



4.2 3.5 
(1.5) (1.8) 



4.5 
(1.4) 



3.9 
(2.0) 



3.8 
(1.9) 



4.2 
(1.6) 



4.6 
(1.5) 



4.6 
(1.6) 



4.0 
(1.7) 

3.8 
(1.6) 



4.3 4.6 4.4 4.9 5.0 4.8 4.6 A. 4 

(1.7) (1.6) (1.4) (1.3) (1.4) (1.6) (1.5) (2.2) 



4.8 5.2 . 4.4 4.1 4.8 3.8 6.5 4.8 

(1.9) (1.8) (1.4) (1.5) (1.1) (1.7) (0.7) (1.1) 
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Table 7 > Total System Freeze 



Mean Ratfngs (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 







ATC 






KAC 






SAC 










TAC 










T-50 


T-51 


T-5 
(EW) 


C-5/ 
C-141 


C-130 


CH-3/ 
HH-53 


FB-lllA FB-niA 
(EW) 


T-4 

(EW) 


B-52 WST F-4E/ 
(EW) F-4G 


F.4G 
(EW) 


F-15 


A-10 


A-10 
(EW) 


E-3A 
FS 


E-3A 
MS 


Frequency 
of Use-R 


5.7 
(K4) 


6.1 

(0.9) 


5.3 
(2.1) 


3.1 
(1.4) 


4.7 
(1.3) 


4.5 
(1.0) 


4.3 
(1.5) 


3.8 
(1.1) 


6.0 
(0.9) 


5.1 

(0.6i 


4.2 
(1.5) 




4.5 
(1.5) 


4.9 
(0.9) 




1.3 
(0.6) 


Frequency 
of Use-C 








3.4 
(1.5) 


4.6 
(1.6) 


6.1 
(0.7) 


4.4 

(1.5) 


4.3 
(1.2) 


6.1 
(1.1) 


4.7 
(1.4) 


4.2 
(1.1) 




3.0 
(1.9) 


5.3 
(0.8) 




1.2 
(0.4) 


Ease 

of Use-R 


6.7 
(0.6) 


6.8 
(0.4) 


6.2 
(0.4) 


6.0 
(0.9) 


6.2 
(0.9) 


6.6 
(0.5) 


6.7 
(0.5) 


6.0 
(1.1) 


6.2 
(0.5) 


6.1 
(0.6) 


6.0 
(0.4) 




5.7 
(0.8) 


6.0 
(0.7) 




4.7 

(1.9) 


Ease 

of Use-C 








5.9 
(0.9) 


6.3 
(0.9) 


6.4 
(0.6) 


6.6 
(0.7) 


6.1 
(0.8) 


6.4 
(0.5) 


6.7 
(0.5) 


6.3 
(0.5) 




5.8 
(1.3) 


6.1 
(0.3) 




2.7 
(2.1) 


Amount of 

Training 

Received-R 


4.6 

(K7) 


5.6 
(1.3) 




3.7 
(1.5) 


4.6 
(1.3) 


4.5 
(1.2) 


4.2 
(1.2) 


4.1 
(1.1) 










4.5 
(1.0) 






1.5 
(0.9) 


Amount of 

Training 

Received-C 








3.6 
(1.5) 


4.8 
(1.0) 


5.4 
(1.4) 


4.3 
(1.5) 


4.4 
(1.4) 










3.8 
(1.9) 






2.0 
(1.1) 


Adequacy of 

Training 

Received-R 






6.6 
(0.5) 












6.6 
(0.7) 


6.3 
(1.0) 


5.6 
(1.1) 






6.3 

(0.9) 






Adequacy of 

Training 

Received-C 


















6.9 
(0.4) 


6.4 
(0.9) 


6.6 
(0.7) 






6.6 

(0.6) 






1 ra 1 III ily 

Value-R 


(0.8) 


(1.0) 


u . o 

(1.3) 


4 4 
(1.4) 


5 6 
(0.9) 


6 0 

n'.z) 


5.5 
(1.6) 


5.5 
(1.1) 


6.6 

(o!8) 


6 6 
(0.7) 


S 4 
(1.0) 




5 2 
(1.2) 


6.1 
(0.9) 




3.2 
(1.6) 


Training 
Value-C 








5.2 
(1.3) 


5.6 
(1.5) 


6.2 
(0.9) 


5.0 
(1.2) 


5.9 
(1.1) 


6.7 
(0.5) 


6.0 
(2.0) 


6.3 
(0.7) 




5.0 
(1.3) 


6.6 
(0.6) 




3.3 
(1.5) 


Potential 

Training 

Value-R 


6.1 
(1.1) 


6.2 
(0.8) 


6.5 
(0.8) 


4.7 
(1.3) 


5.8 
(0.9) 


6.3 
(1.2) 


5.5 
(1.4) 


5.9 
(1.0) 


6.6 
(0.8) 


6.7 2.6 
(0.5) (2.0) 


5.8 
(l.I) 


4.0 
(2.1) 


5.3 
(1.3) 


6.2 
(0.9) 


5.6 
(0.5) 


3.6 
(1.5) 


Potential 

Training 

Value-C 








5.6 
(0.9) 


5.8 
(1.0) 


6.2 
(1.5) 


5.9 
(1.0) 


6.0 
(0.9) 


6.4 
(0.9) 


6.7 4.2 
(0.5) (2.1) 


6.5 
(0.7) 


5.3 
(1.0) 


4.9 
(1.7) 


6.5 
(0.9) 


2.0 
(1.4) 


3.8 
(1.6) 



Table 8. Reset 



Mean. Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (») and CTU (C) Simulator Instructors 



ATC 



MAC 



SAC 



T-50 T-51 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 

of Use-R 

of Use-C 

Amount of 

Training 

Received-R 

Amount of 

Training 

Received-C 

Adequacy of 
Traini ng 
Received-R 

Adequacy of 

■""raining 

Received-C 

Training 
Value-R 

Training 
Value-C 

Potential 

Training 

Value-R 

Potential 

Training 

Value-C 



5.6 
(1.3) 



5.9 
(0.8) 



4.2 
(1.1) 



T-5 
(EW) 



C-5/ C-130 
C-141 



CH-3/ 
HH-53 



FB-lllA FB-lllA T-4 B-52 WST 
(EW) (EU) (EW) 



TAC 



F-4E/ 
F-4G 



6.2 4.5 
(0.8) (1.5) 



6.0 5.9 
(0.7) (0.5) 



5.1 
(0.9) 



6.1 
(1.3) 



5.9 6.3 6.4 
(0.9) (0.8) (0.8) 



6.3 6.2 6.6 
(0.7) (0.7) (0.6) 



4.5 5.7 

(1.9) (1.0) 

4.3 5.5 
(1.9) (1.3) 

5.4 5.5 
(0.8) (1.0) 

5.5 5.9 
(1.0) (0.8) 



3.7 4.6 
(1.7) (1.2) 



3.6 4.8 
(1.8) (0.8) 



5.2 5.5 
(1.2) (1.0) 



5.6 5.4 
(1.1) (1.6) 



5.7 
(0.8) 



6.0 
(1.0) 



4.3 
(1.3) 

5.5 
(1.1) 

5.2 
(0.9) 

5.6 
(0.8) 



4.1 
(1.0) 



5.0 
(1.6) 



5.0 
(1.3) 



5.5 6.0 5.6 
(1.1) (1.2) (1.3) 



5.7 
(1.0) 



5.5 
(1.7) 









4.U 


5.2 


(0.8) 


(1.3) 


(1.0) 


(1.5) 


(Kl) 


4.5 


5.1 


5.5 


4.2 


5*0 






1 1 •2) 


1 1 ♦2) 


(1 .3) 


6 2 




7 


1 

9« 1 


C Q 


(0.8) 


(1.6) 


(0.9) 


(1.1) 


(1.2) 


5.6 


5.7 


5.9 


5.3 


6.0 


V 1 • 1 / 








I 1 . 1 ) 


4.2 


3.6 






3.7 


(K5) 


(1.2) 






(1.4) 


4.3 


4.3 






4.0 










1 1 . u 






6.6 


4.5 










11*// 








6.5 


().0 










I 1 « 3 1 




5.5 


4.6 


6.3 


4.9 


5.4 


(1.2) 


(1.6) 


(1.2) 


(1.8) 


(1.1) 


5.0 


5.4 


6.6 


5.7 


5.4 


(1.6) 


(1.1) 


(0.5) 


(1.8) 


(1.3) 


5.8 


5.3 


6.0 


6.1 


5.5 


(1.2) 


(0.9) 


(1.1) 


(0.6) 


(0.9) 


5.6 


S.7 


6.5 


6.2 


5.4 


(1.2) 


(0.9) 


(0.5) 


(1.6) 


(1.2) 



F-4G 
(EW) 



F-15 A-10 



A-10 
(EW) 



E-3A 
FS 



E-3A 
MS 



4.0 4.8 5.0 3.3 6.2 

(1.4) (1.6) (1.2) (1.9) (0.8) 

4.9 2.8 4.3 5.1 4.7 

(0.9) (2.0) (1.4) (0.6) (1.2) 

5.5 5.3 5.1 4.7 6.4 

(0.9) (1.0) (0.7) (1.3) (0.9) 

6.0 4.7 5.4 5.9 5.7 

(0.9) (1.3) (0.9) (0.7) (0.6) 



4.0 4.3 
(1.4) (1.0) 



1.9 4.2 
(1.1) (1.3) 



4.4 

(1.5) 



4.3 

(2.1) 



5.2 
(1.5) 



6.4 
(0.8) 



5.1 
(1.9) 



6.2 
(1.1) 



5.5 5.6 5.2 4.1 5.8 

(0.9) (0.8) (1.2) (1.5) (0.8) 

6.1 3.5 5.3 5.8 5.0 

(1.0) (1.4) (1.4) (1.3) (0.0) 



6.0 5.4 5.2 5.1 5.8 3.6 
(0.8) (1.0) (1.2) (1.8) (0.4) (1.8) 



6.4 4.8 5.8 6.2 5.5 5.4 
(0.6) (1.9) (0.9) (0.9) (2.1) (0.9) 
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utility and utilization . It makes sense to consider total system 
freek:e and reset together, since they were so often used in conjunction. 
Sis used total system freeze when they wished to temporarily suspend the 
training mission in order to provide the student with instruction or 
feedback* They then used reset in order to restore the mission. There 
were few problems associated with the use of these features. The 
frequency- and ease-of-use ratings indicated that these features tenaed to 
be implemented very easily and with moderate regularity. The only 
exceptions occurred at sites in which (a) it was necessary to reinitialize 
the ATD in order to resume training following the use of total system 
freeze (e.g., E-3A mission simulator), or (b) the use of reset was not 
sufficient to permit resumption of the mission from the point at which it 
had been suspended (e.g., F-15 simulator). 

Comparisons across HAJCQMs and levels of training . The EW Sis' 
utility ratings of total system freeze and reset were high, and did not 
differ across the MAJCQMs. There was more variability in the non-EW Sis' 
ratings; utility was highest for ATC Sis, lowest for TAC Sis. 

On several ATUs, there were significant differences in utility and 
utilization across the two revels of training. Most of these differences 
showed higher ratings by RTU Sis. On the F-15 ATD, the RTU Sis used reset 
more frequently than did the CTU Sis. They also received more training in 
the use of reset and rated the feature as having greater training value. 
On the A-10 ATD, the RTU Sis used total system freeze more often than did 
the CTv^ Sis; and on the E-3A flight simulator, the RTU Sis rated the 
feature higher in potential training value than did the CTU Sis. However, 
on the C-S/C-Ul ATD, the CTU Sis assigned higher training value ratings 
to total system freeze than did the RTU Sis. 

Crash/Kill Override (Table 9) 

Incidence . Crash/kill override was surveyed at every non-EW ATD 
site. It was available for use on all non-EW devices. 

Utility and utilization . We can acc.>tmt for the generally high 
ratings by considering that although crash/kill override is more properly 
viewed as a variation of the task difficulty feature, it was more often 
used for training management. If "crashes" or "kills" were permitted to 
occur, a tedious reinitialization of the ATD would typically be required. 
Thus, the feature was used, more often thar< not, in o/*der to avoid the 
loss of instruction time. 

Comparisons across HAJCOMs and levels of training . "Crashes" were 
most often permitted (I.e., crash/kill override was not activated) on the 
ATC ATDs (viz., T-50, T-51) and on TAC^s E-3A mission simulator. On the 
C-5/C-141 ATD, the RTU Sis used the feature more often than did the CTU 
Sis; and on the F-15 ATD, the RTU Sis rated the feature as easier to use 
than did the CTU Sis. ^ 
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Table 9 . Crash and/or Kill Override 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
T . 2 u^,^ °^ Use, Amount of Training Received, Adequacy of Tr&ining Received. 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



SAC 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 
of Use-R 

Ease 

of Use-C 

Amount of 

Training 

Received-R 

Amount of 

Trainini? 

Received-C 

Adequacy of 

Training 

Received-R 

Adequacy of 

Training 

Received-C 



T-50 



T-51 T-5 
(EW) 



3.6 3.8 
(1.4) (1.4) 



5.7 5.8 
(1.2) (1.3) 



3.7 4.0 
(1.3) (1.3) 



C-5/ 
C-141 



C-130 



CH-3/ 
HH-53 



FB-lllA FB-lllA T-4 B-52 WST 
(EW) (EW) (EW) 



TAC 



5.5 5.1 

(2.1) (1.4) 

4.4 5.0 

(2.2) (1.4) 

6.3 6.0 

(0.9) (1.0) 

6.1 6.3 

(1.0) (0.9) 



5.1 
(2.0) 

5.4 
(1.6) 

6.5 
(0.7) 

6.1 
(0.8) 



3.6 4.7 4.1 
(1.6) (1.5) (1.8) 



3.6 4.8 4.8 
(1.5) (1.0) (1.8) 



4.8 
(1.3) 

5.5 
(1.4) 

6.3 
(0.8) 

6.2 
(1.0) 



4.3 
(1.2) 



4.5 
(1.4) 



F-4E/ 
F-4G 



F-4G 
(EW) 



F-15 A-10 



A-10 
(EW) 



E-3A 
FS 



E-3A 
MS 



6.4 


5.6 
U ./) 


4.6 

(2.1 ) 


4.6 
(1 .8) 


3.1 
(2.4) 


6.5 
(0.7) 


5.1 
(1.3) 


3.8 

(1 9) 


5.0 
11./; 


3.0 


6.5 
(0.7) 


6.7 
(0.7) 


5.9 
(1.1) 


5.8 
(1.8) 


5.8 
(1.3) 


6.5 
(0.7) 


5.8 
(0.9) 


5.6 
(1.4) 


5,3 
(2.1) 


6.0 
(1.0) 


3.2 
(1.8) 


4.3 
(1.1) 


4.2 
(1.5) 


5.4 
(1.5) 


2.8 
(2.1) 


3.8 
(1.3) 


3.8 
(1.5) 


3.8 
(1.6) 


3.3 
(2.1) 


2.7 
(2.0) 



Training 
Value-R 

Training 
Value-C 

Potential 

Training 

Value-R 

Potential 

Training 

Value-C 



4.4 4.4 
(1.5) (1.5) 



4.8 4.8 
(1.4) (1.1) 



4.6 5.3 5.1 
(1.7) (1.1) (1.4) 



4.8 5.1 
(1.7) (1.1) 



4.9 5.5 
(1.5) (1.1) 



4.8 
(1.9) 



5.5 
(1.3) 



5.0 5.8 4.8 
(1.7) (0.8) (1.8) 



4.7 
(0.5) 

5.1 
(1.6) 



5.0 
(1.7) 



5.5 
(1.3) 



5.4 

(1.5) 

4.8 
(1.5) 



5.5 
(1.4) 



5.2 
(1.8) 



5.3 
(1.3) 



4.9 
(1.6) 



4.6 5.0 
(1.2; (1.4) 



5.4 4.6 
(1.4) (1.7) 



4.7 4.8 
(1.4) (1.7) 



5.4 4.7 
(1.5) (2.0) 

4.7 4.3 
i'0.6) (0.6) 



5.4 3.3 
(1.1) (2.1) 



4.0 3.4 
(0.0) (1.7) 



ERIC 
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Automated Adaptive Training (Table 10) 

Incidence , Automated adaptive training was surveyed at every non-EW 
1^ ATD. site; It was available for rre only on the F-4E ATD, 

utility and utilization . Utilization of automated adaptive training 
was minimal. The F-4E Sis' ratings of the frequency of use, ease of use, 
and amount of training received were lower than 95% of the comparable 
ratings of the other training management features. According to the Sis 
who used, automated adaptive training (the majority did not use it and more 
than .20* were unaware of its availability), there were two major problems 
with the feature: The first was that "it takes the instructor out of the 
loop"; the second was that it could be applied only at certain points 
during a mission, 

* * 

Comparisons across the MAJCOMs and levels of training . The potential 
training value ratings were statistically equivalent across the MAJCOMs 
and between the two levels of training, with one exception: The E-3A 
flight simulator RTU Sis' mean rating was significantly higher than that 
of the CTU Sis, 

Programmed Mission Scenarios and Manual Mission Control (Tables 11 and 12) 

Incidence, Programmed mission scenarios were surveyed at every ATD 
site. They were available for use on all but the T-50, T-51, C-5/C-141, 
and CH-3/HH-53 ATDs, Manual mission control was surveyed during the EW 
phase of the project. Data were collected from each of the EW trainers 
except the FB-lllA, 

Utility and utilization . Sis' ratings of the utility and utilization 
of programmed mission scenarios varied greatly across training devices. 
For example, in non-EW applications, frequency of use ranged from 1,0 
(A-10 ATD> RTU) to 6,0 (E-3A flight simulator, CTU), In general, the most 
favorable ratings were obtained from the T-5, T-4, and B-52 WST Sis, 

There were two important factors that limited the use of programmed 
mission scenarios. First, a typical simulated mission consisted of a long 
and complex sequence of events, and the programming of scenarios was thus 
a tedious and difficult task. Not surprisingly, there was an insufficient 
number of scenarios to accomplish training at most sites. Those that were 
available were frequently characterized as "unreliable," "limited," or 
"outdated," Second, approximately 25« of the C-130, F-4E/F-4G, F-15, and 
A-10 Sis commented that they preferred the increase in instructional 
flexibility afforded by manual mission control. Unfortunately, ratings of 
manual mission control were only obtained from EW Sis, and these ratings 
closely matched those of programmed mission scenarios, with one exception: 
T-5 Sis greatly preferred programmed to manual scenarios. 
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Table l6 > Automated Adaptive Training 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 

of Use-R 
Ease 

of Use-C 

Amount of 

Training 

Received-R 

Amount of 
Training 
Recei ved-C 

Adequacy of 

Training 

Received-R 

Adequacy of 
Training 
Recei ved-C 



MAC 



SAC 



T-50 



T-51 T-5 
(EW) 



C-5/ C-130 
C-141 



CH-3/ 
HH-53 



FB- 



lllA FB-lllA 
(EW) 



T-4 
(EW) 



B-52 WST 
(EW) 



TAC 



F-4E/ 
F-4G 



F-4G 

(EW) 



F-15 A-10 



A-10 

(EW) 



E-3A 
FS 



E-3A 

MS 



1.4 

(0.8) 

2.7 
(1.5) 

2.4 
(1.1) 

3.5 
(0.7) 



1.3 

(0.6) 



2.7 
(2.1) 



Training 
Value-R 

Training 
Value-C 

Potential 

Training 

Value-R 

Potential 

Training 

Value-C 



4.1 4.0 
(2.0) (IJ) 



3.8 3.5 
(1.8) (2.0) 



4.2 3.7 
(1.9) (2.2) 



3.2 
(2.0) 



3.6 
(2.0) 



4.3 

(2.0) 



4.7 
(1.6) 



4.0 
(0.9) 

4.3 
(1.5) 



4.6 
(1.3) 



4.0 
(1.0) 



4.2 4.0 
(1.7) (1.9) 



3.6 3.7 
(1.5) (1.4) 



4.4 5.0 
(1.7) (1.9) 



1.0 3.8 
(0.0) (K9) 



.ERIC 
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Table IK Programmed Mission Scenarios 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



SAC 



TAC 



00 



T-50 T-51 T-5 



C-6/ C-130 CH-3/ FB-lllAFB-niA T-4 B-52 WST F-4E/ F-4G F-15 A-10 A-10 E-3A E-3A 









(EW) 


C-141 




HH-53 




(EW) 


(EW) 


(EW) 


F-4G 


(EW) 






(EW) 


FS 


MS 


Frequency 
of Use-R 






6.4 
(1.1) 




2.0 
(1.9) 




3.5 
(2.5) 


3.7 
(2.1) 


5.9 
(1.4) 


5.1 
(2.1) 


2.1 
(2.1) 


3.0 
(1.8) 


2.1 
(1.2) 


1.0 
(0.2) 


4.3 

(2.2) 


1.4 
(0.9) 


3.4 
(1.8) 


Frequency 
of Use-C 










1.5 
(0.7) 




5.1 

(1.8) 


5.4 

(1.7) 


6.2 
(1.2) 


6.7 
(0.8) 


3.3 
(2.2) 


4.1 
(1.7) 


4.1 
(1.7) 


1.5 
(0.8) 


3.0 
(1.7) 


6.0 
(1.0) 


3.5 
(1.4) 


Ease 

of Use-R 






6.2 
(1.1) 




4.7 
(1.1) 




4.2 
(2.4) 


4.4 
(1.3) 


fa.O 
(1.2) 


6,.l 
(1.3) 


2.9 
(1.5) 


5.7 
(1.2) 


4.9 
(1.4) 


1.0 
(0.0) 


4.9 

(2.0) 


3.0 
(0.0) 


3.7 
(1.8) 


Ease 

of Use-C 










5.2 
(1.5) 




4.0 
(1.3) 


4.5 
(1.4) 


5.9 
(2.1) 


6.7 
(0.5) 


4.1 
(0.9) 


5.9 
(1.1) 


4.7 
(1.1) 


2.8 
(1.7) 


4.5 
(1.6) 


5.0 
(1.4) 


4.2 
(1.3) 


Amount of 

Training 

Received-R 










2.9 
(1.8) 




3.3 
(2.1) 


3.1 
(1.4) 






1.8 
(1.8) 




3.0 
(1.8) 


1.3 
(0.9) 




2.2 
(■^.7) 


2.9 
(1.7) 


Amount of 

Training 

Received-C 










3.5 
(1.4) 




3.3 
(1.9) 


4.7 
(1.6) 






2.6 
(1.6) 




2.8 
(1.4) 


2.3 
(1.6) 




3.7 
(0.6) 


4.0 
(1.3) 


Adequacy of 

Training 

Received-R 






6.0 
(1.6) 












6.3 
(1.0) 


5.4 
(1.8) 




5.0 
(1.3) 






5.0 
(1.7) 






Adequ<jcy of 
Training 
Kece 1 vsu-i* 


















6.5 


5.8 

(1 S) 




5.6 

(1 6) 






4.0 

(1.8) 






Training 
Value-R 






6.6 

(0.7) 




4.8 
(1.6) 




3.8 
(2.4) 


4.6 
(1.7) 


6.4 
(0.7 


5.8 
(1.4) 


4.4 
(1.7) 


4.5 
(2.0) 


4.9 
(1.4i 


2.8 
(1.2) 


3.6 
(2.1) 


4.U 

(0.0) 


4.6 
(2.1) 


Training 
Value-C 










3.7 
(1.3) 




4.6 

(1.6) 


6.5 

(1.4) 


6.8 
(0.5) 


(0.8) 


5.0 
(1.4) 


5.1 

(1.4) 


5.'! 

(1.0; 


(l.e) 


3.2 
(1.4) 


4.7 
(1.2) 


5.0 
(0.6) 


Potential 

Training 

Value-R 


4.8 
(1.8) 


4.3 
(1.0) 


6.4 
(0.9) 


4.3 
(1.6) 


5.1 
(1.6) 


4.5 
(2.3) 


5.0 
(1.7) 


5.5 
(1.5) 


6.4 
(0.7) 


6.8 

(0.4) 


4.9 
(1.5) 


b.4 
(1.2) 


b.2 
(1.5, 


4.3 
(1.9) 


5.5 
(1.5) 


(0.4) 


6.1 
(1.3) 


Potential 

Training 

Value-C 








4.6 
(1.4) 


4.9 
(2.1) 


4.5 
(1.9) 


5.2 
(1.4) 


5.7 
(1.3) 


6.8 
(0.5) 


6.7 
(0.3) 


5.0 
(1.7) 


b.4 
(1.3) 


5.C 
(1.3) 


5.U 
(1.1) 


4.8 
(1.4) 


4.U 
(1.4) 


4.8 
(1.3) 



Table 12 > Manual Mission Control 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



vo 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 

of Use-R 
Ease 

of Use-C 

Afnount of 

Training 

Received-R 

Anount of 

Training 

Received-C 



T-50 



T-51 T-5 
(EW) 



1.6 
(0.8) 



3.9 

(M) 



SAC 



C-S/ C-130 
C-141 



CH-3/ 
HH-53 



FB-lllA FB-lllA T-4 B-52 WST 
(EW) (EW) (EW) 



TAC 



F-4E/ 
F-4G 



F-4G 
(EW) 



F-15 A-10 



6.0 5.8 
(1.1) (1.6) 

6.1 5.3 
(0.8) (0.8) 

4.7 4.1 

(1.1) (0.9) 

6.0 4.3 

(0.5) (1.4) 



6. 
(0. 



5.6 
(1.2) 

4.2 
(1.0) 

4.3 
(1.1) 



A-IO 
(EW) 



E-3A 
FS 



E-3A 
MS 



3.5 
(2.4) 

3.1 
(2.3) 

2,9 

(1.7) 

2.6 
(1.0) 



Adequacy of 

Training 

Received-R 

Adequacy of 

Training 

Received-C 

Training 
Value-R 

Training 
Value- C 

Potential 

Training 

Value-R 

Potential 

Training 

Value-C 



3.2 
(1.4) 



2.9 
(1.6) 



4.5 

(1.6) 



6.2 5.0 
(1.1) (1.8) 



6.4 6.2 
(0,5} (1.0) 

6.7 6,6 
(0,6) (0,7) 

6,6 6,5 
(0,5) (0,8) 



6,6 7,0 
(0,6) (0,0) 



6,9 6,7 
(0,4) (0.5) 



4,9 
(1,5) 



5,5 
(1,7) 

6.3 
(1,1) 

6,4 
(0,8) 



6,4 
(0.7) 



6.5 
(0,5) 



3,8 
(1,7) 



4.0 

(1,8) 

4,0 
(2,1) 

3,7 
(1.8) 



0,2 
f0,8) 



5,8 
(1,3) 



;ERIC 
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Comparisons across MAJCOMs and levels of training . Although there 
were no obvious differences in the ratings of programmed mission scenarios 
across MAJCOMs,, the feature appeared to be more important for CTU 
trainins. Thus, FB-lllA, FB-lllA (EW), F-15, and E-3A flight simulator 
CTU Sis ^used the feature significantly more often than did their RTU 
counterparts. Moreover, A-10 CTU Sis rated programmed mission scenarios 
higher In training value than did A-10 RTU Sis, The ratings of manual 
mission control did not differ across the two levels of training. 
Comparisons across MAJCOMs were precluded due to a lack of data. 

Variation of Task Difficulty/Fidelity Features 

A utomated Malfunction Insertion (Table 13) 

Incidence , Automated malfunction insertion was surveyed at every ATD 
site. It was available for use on every device exxept the C-5/C-141, 
CH-3/HH-53, and FB-lllA ATDs, and the E-3A mission simulator. 

Utility and utilization . The ratings of this feature varied 
greatly^ On two devices (viz,, E-3A flight simulator, B-52 WST) it worked 
well, and the frequency of use was higher than it was on the remaining 
devices. For most of those ATUs, especially the F-4E/F-4G, F-15, and 
A-IO, it was said that it was time-consuming to implement and unreliable, 
and did not always reflect mission profiles. Moreover, almost 20% of the 
Sis commented that they preferred to insert malfunctions manually. There 
was a clear parallel between the utility and utilization of automated 
malfunction insertion and that of programmed mission scenarios. The 
operational difficulties and limitations of both features restricted their 
use, and a substantial number of Sis preferred the benefits of manual 
malfunction insertion and manual mission control, respectively, 

Coiiiparisons across MAJCOMs and levels of training . Table 13 clearly 
suggests that the utMity and utilization of automated malfunction 
insertion was lowest for TAC fighter ATDs, The only significant 
comparisons across the two levels of training were for the C-130 ATD, The 
feature was used more often and was rated higher in training value by RTU 
than by C^U Sis, 

Environmental (Table 14) 

Incidence , Environmental was surveyed at every non-EW ATD site. The 
capability was available for use on every device except the FB-lllA ATD, 

Utility and utilization . The mean racings of the environmental 
feature were uniformly in the moderate to high range of the scales, with 
one exception: the E-3A mission simulator. The favorable ratings of this 
feature were apparently due to its easy, reliable operation and its value 
in training instrument flying under adverse weather conditions. Frequency 
of use was significantly lower on the E-3A mission simulator. 
Environmental simulation on this device was limited to "winds aloft," 
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Table 13 > Automated Malfunction Insertion 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 

of Use-R 
Ease 

of Use-C 

Amount of 

Training 

Received-R 

Amount of 

Training 

Received-C 

Adequacy of 

Training 

Received-R 

Adequacy of 

Training 

Received-C 

Training 
Value-R 

Training 
Value-C 

Potential 

Training 

Value-R 

Potential 

Training 

Value-C 



T-50 T-51 



3.7 
(2.2) 



5.2 
(1.7) 



3.3 
(1.7) 



5.0 
(1.6) 



5.4 
(1.5) 



T-5 
(EW) 



C-5/ €-130 CH-3/ 
C-141 HH-53 



SAC 



TAC 



FB-lllA FB-lllA 
(EW) 



3.1 3.4 
(1.9) (1.6) 



5.3 5.5 
(1.0) n.7) 



3.5 
(1.6) 



5.3 
(2.1) 



4.1 4.8 
(2.1) (1.6) 



4.7 4.9 
(1.8) (1.7) 



3.9 
(2.8) 

1.5 
(0.5) 

5.0 
(1.9) 

5.8 
(0.5) 



3.8 
(2.1) 



3.5 
(1.5) 



5.2 
(1.8) 

3.1 
(1.6) 



4.7 5.6 5.4 
(1.6) (1.5) (1.6) 



4.8 4.8 5.6 
(1.3) (2.0) (1.7) 



4.2 

(2.3) 



4.7 
(1.7) 



5.5 
(1.7) 



5.2 
(1.2) 



T-4 


8-52 WST 


F-4E/ 


F-4G 


F-15 


A-10 


A-10 


E-3A E-3A 


(EW) 


(EW) 


F-4G 


(EW) 






(EW) 


FS MS 


4.5 


5.1 


1.4 




3.0 


2.6 


1.4 


5.4 


(1 0) 


(1 1) 


(1 31 




1 1 . i 


I 1 .Oi 


1 1 .uj 


1 1 . 


4.3 


4 0 


1 n 




1 7 


c . D 


1 


A 'J 


(3.1) 


(2!7) 


(0.0) 


(1.7) 


(1.7) 


(1.6) 


(1.2) 


(1.5) 


5.6 


5.7 


3.0 




4.9 


3.5 


3.2 


4.2 


(0 5^ 


(1 3^ 








\t •O) 


1 1 .0 J 




4.7 






o 


A. A. 
•t. 


c. 0 


A 1 




(r.2) 


(0.8) 


(0.0) 


(2.0) 


(1.2) 


(1.4) 


(1.5) 


(2.0) 






1 2 






J.J 










(o!6) 




(1.8) 


(1.7) 




(2.1) 






1.2 




3.6 


3.6 




4.3 










M 0\ 
U • ci 


(1 .bj 




(0.5) 


5.5 












A 0 




(K5) 


(2.2) 














6.7 


6.0 




4:0 






4.0 




(0.6) 


(1.6) 




(2.4) 






(2.0) 




5.5 


5.9 


4.2 




4.9 


3.6 


2.8 


5.6 


(1.5) 


(0.9) 


(1.3) 




(1.7) 


(1.7) 


(1.7) 


(1.5) 


6.5 


5.5 


2.5 


2.8 


4.0 


4.5 


3.2 


4.7 


(0.6) 


(2.0) 


(0.7) 


(2.1) 


(1.8) 


(1.6) 


(0.9) 


(1.5) 



4.8 

(1.8) 



6.2 
(0.7) 



5.5 6.8 
(1.7) (0.4) 



4.6 4.3 4.8 4.4 4.2 5.8 3.5 
(1.6) (1.7) (1.4) (1.8) (1.8) (0.8) (1.8) 



3.4 4.4 4.5 4.9 4.0 5.0 4.0 
(1.8) (1.8) (1.3) (1.2) (1.2) (0.0) (2.0) 
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Tible 14, Envi ronmcntal 



Mean Ratings (and Standard Deviations) of the Frequency of Use» 
Ease of Use» Amount of Training Received^ Adeauacy of Training Received^ 
Training Value* and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC MAC ^ SAC ^ TAC 





l-DU 




(EU) 


C-141 


L-l ou 


HH-53 


F8-111A FB-lllA 
(EU) 


T A 

(EU) 


0 C9 UCT 

(EU) 


C AC / 

r-Ht/ 
F-4G 


c Ar 
(EU) 


C 1 c 

r -1 b 


A-lU A-lO 

(EU) 


t-oA 
FS 


t-JA 
MS 


Frequency 
Of use-K 


5.5 
(1.3) 


6.3 
lO./J 




5.1 

11. 0) 


5.4 
(1.0) 


4.8 

'1 0\ 

1 .2) 










4.1 

(1.5) 




4.2 
(1.9) 


4.5 
(1.2) 


6.0 
(1.2) 


1.9 
(1.5) 


Frequency 
of Use-C 








5.3 
(1.7) 


6.4 
(0.9) 


6.2 
(1.0) 










3.8 
(1.9) 




3.1 
(2.0) 


3.0 
(1.7) 


6.3 
(1.2) 


2.3 
(1.5) 


Ease 

of Use-R 


5.2 
(1.3) 


5.1 
(1.3) 




5.3 
(0.8) 


5.4 
(1.2) 


5.3 
(0.6) 










5.1 
(1.2) 




5.4 
(1.2) 


4.3 
(1.0) 


5.4 
(1.7) 


4.6 
(1.5) 


base 

of Use-C 








(0.8) 


(1.0) 


c c 
5,0 

(1.0) 










5.3 
(1.3) 




4.1 
(1.3) 


4.8 
(1.2) 


6.0 
(1.0) 


4.5 
(0.7) 


^nount of 
1 raining 
Recelved-R 


(1.6) 


A 7 

(1.1) 




(1.3) 


A Q 

(1.4) 


7 Q 
J. I' 

(1.6) 










Z.B 
(1.3) 




4.0 
(1.3) 


4.2 
(1.1) 


5.2 
(1.6) 


1.8 
(1.3) 


Training 
Recelved-C 








3.7 
(1.6) 


4.8 
(0.6) 


5.4 
(1.4) 










3.2 
(1.5) 




2.9 
(1.0) 


3.4 
(1.3) 


5.0 
(0.0) 


2.7 
(1.4) 


Adequacy of 

Training 

Recelved-R 


































Adequacy of 

Training 

Recelved-C 






























• 




Training 
Yalue-R 


6.2 
(0.9) 


6.2 
(1.0) 




5.2 
(1.3) 


5.4 
(1.4) 


5.2 
(1.5) 










4.9 
(1.1) 




4.2 
(1.2) 


5.1 
(1.2) 


5.4 
(0.9) 


3.3 
(1.4) 


Training 
Value-C 








5.3 
(1.3) 


5.9 
(1.0) 


5.8 
(1.0) 










4.1 
(2.3) 




3.6 
(1.1) 


4.0 
(1.2) 


6.0 
(1.0) 


3.3 
(1.5) 


Potential 

Training 

Value-R 


6,3 
(1.0) 


6.3 
(1.0) 




5.5 
(1.1) 


5.7 
(1.1) 


6.2 
(1.2) 


5.3 
(0.8) 








5.0 
(1.2) 




4.8 
(1.2) 


5.6 
(1.3) 


5.8 
(0.4) 


2.8 
(2.0) 


Potential 

Training 

Valu?*C 








5.9 
(0.8) 


6.2 
(0,6) 


5.9 
(1.6) 


5,7 
(0.9) 








5.1 
(1.9) 




3.9 
(1.3) 


4.6 
(1.4) 


4.0 
(2.8) 


2.4 
(0.9) 



Comparison across HAJCOMs and levels of training . There were no 
significant differences in the utility and utilization of the 
environmental feature across t^JCOMs. Howevarj there were several 
significant differences across the two levels of training. A-10 RTU Sis 
used it more often and rated it higher in training value than did A-10 CTU 
Sis; and F-15 RTU Sis rated it easier to use and received more training in 
its use than did F-15 CTU Sis. 

Motion (Table 15) 

Incidence . Platform motion was the only form of motion cueing 
surveyed. It was surveyed at every non-EW ATD site. It was available for 
use on the T-50, T-51, C-5/C-141, C-130, CH-3/HH-53, FB-lllA, and F-15 
ATDs, and the E-3A flight simulator. 

Utility and utilization . Except for the F-15 ATU, the utility and 
utilization ratings of operational platform motion cueing systems were 
uniformly in the high range of the rating scales. Over 50% of the F-15 
Sis commented that the motion simulation was unrealistic and wouid not 
yield positive transfer of training to aircraft itself. 

Comparisons across MAJCOMs and levels of traininci . In general, the 
utility Uctual and potential) and utilization ratings of ATC, MAC, and 
SAC motion systems were clearly higher than those of TAC. A more accurate 
statement would be that non-fighter aircraft ATD platform motion systems 
were evaluated more favorably than were fighter aircraft ATD platform 
motion systems, since TAC's E-3A flight simulator motion system received 
extremely high ratings. 

There were several significant differences in the ratings across the 
two levels of training. The F-15 KTU Sis rated motion higher in training 
value and potential training value than did the F-15 CTU Sis, whereas the 
C-5/C-141 and A-10 CTU Sis rated motion higher in training value and 
potential training value, respectively, than did the RTU Sis. 

Partial Freeze (Table 16) 

Inc^'dence . Partial freeze was surveyed at every ATD site. It was 
available tor use on every device except the T-5 and T-4 ATDs, and the 
E-3A mission simulator. 

Utility and utilization . As with total system freeze, the ratings of 
partial freeze Indicated that it tended t'- be implemented very easily and 
with moderate regularity. Although it ; jperly considered a variation 

of the task difficulty feature, partial freeze was used in a manner 

similar to crash/kill override (see above); that is, to manage training. 
In fact, over 80% of Sis occasionally used partial freeze as a substitute 
for total system freeze in order to temporarily suspend the training 
session and instruct the student. Partial freeze appeared to offer two 
advantages over total system freeze for this purpose. First, on certain 
devices (e.g., C-5/C-141, FB-lllA) it was simply easier to reinitialize 
the ATD following a partial rather than total freeze. Second, by freezing 
only particular flight parameters. Sis were able to instruct while still 



Table 15. Motion 



Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 







ATC 




MAC 






SAC 








TAC 








T-50 


T-51 T-5 
(EW) 


C-5/ 
C-141 


C-130 


CH-3/ 
HH-S3 


FB-lllA FB 


-lllA T-4 
(EW) (EW) 


B-52 WST 
(EW) 


F-4E/ 
F-4(i 


F-4G F-15 
(EW) 


A-10 A-10 
(EW) 


E-3A 
FS 


E-3A 
MS 


Frequency 
of Use-R 


6.6 

(K2) 


6.2 
(1-9) 


5.1 
(2-0) 


6.6 
(K4) 


5.3 
(2.4) 


5.5 
(2.0) 








2.5 
(l.b) 




6.4 
(1.3) 




Frequency 
of Use-C 






5.7 
(2.1) 


7.0 
(0.0) 


6.6 
(0.9) 


5.9 
(1.8) 








1.5 
(0.7) 




7.0 
(0.0) 




Ease 
of Use-R 


6.4 
(0.6) 


6.4 
(0.7) 


5.7 
(1.0) 


5.8 
(1.1) 


6.3 
(0.9) 


5.3 
(1.8) 








5.4 

(1.2) 




6.2 
(1.3) 




Ease 

of Use-C 






5.9 

(L3) 


6.2 
(1.0) 


6.2 
(1.0) 


6.2 
(1.0) 








4.4 

(2.0) 




6.3 
(O.b) 




Amount of 

Training 

Received-R 


4.6 

(1-6) 


4.8 

(2.0) 


3.6 

(K5) 


5.0 
(1.6) 


4.7 

(2.2) 


4.7 

(0.6) 








3.4 
(1.7) 




5.4 
(1.5) 




Amount of 

Training 

Received*C 






3.8 
(1.6) 


5.5 
(1.1) 


5.5 
(1.9) 


4.8 
(1.7) 








2.4 
(1.3) 




6.0 
(1.0) 




Adequacy of 

Training 

Received-R 




























Adequacy of 

Training 

Received-C 




























Training 
Value-R 


6.1 

(KO) 


6.0 

(KO) 


5.4 

(K3) 


6.2 
(1.3) 


6.2 
(1.4) 


5.2 
(1.2) 








3.6 
(l.fa; 




6.0 
iO.4) 




Training 
Value-C 






6.1 
(Kl) 


6.4 
(0.7) 


6.4 
(1.1) 


*..6 
(2.0) 








2.3 
(1.2. 




6. J 

(1.2) 




Potential 

Training 

Value-R 


6.3 

(Kl) 


5.9 

(Kl) 


5.5 

(K3) 


5.9 
(1.3) 


5.7 
(2.1) 


5.7 
(1.6) 






3.2 
(1.5) 


4.2 
{1.7; 


3.0 
(2.1) 


6.8 

:o.4) 


l.y 
(1.4) 


Potential 

Training 

Value-C 






6.3 
(Kl) 


6.6 
(0.8) 


6.5 
(1.5) 


5.2 
(1.9) 






3.8 
(2.0) 


2.4 
(1.4) 


5.1 
(1.4) 


7.0 
iO.O) 


1.2 

(0.4) 



CO 



Table 16 > Partial Freeze 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



( ERiC 





T-50 


T-51 


T-5 
(EW) 


C-5/ 
C-141 


C-130 


CH-3/ 
HH-63 


FB-lllA FB-lllA 
(EW) 


T-4 
(EW) 


D^C No 1 

(EW) 


C AC / 

F-4U 


C AC 

(EW) 


F-15 


1 

A-lO 


A-10 
(EW) 


E-3A 
FS 


E-3A 

MS 




Frequency 
cf Use-R 


3.9 
(1.7) 


5.3 
(1.4) 




4.3 
(1.8) 


4.7 
(1.5) 


4.3 
(1.7) 


5.7 
(1.0) 


5.3 
(1.2) 




(1.3) 


0. 1 
(1.5) 


4.4 
(0.9) 


6.3 
(0.8) 


2.6 
(1.8) 


2.1 
(1.8) 


5.9 
(1.5) 






Frequency 
of Use-C 








3.1 
(2.1) 


4.1 
(1.4) 


4.7 
(1.9) 


4.8 
(1.5) 


4.3 
(1.7) 




3.2 

\ 1*0) 


3.4 

M C A 

U .b) 


3.9 
[ 1 .4) 


3.9 
(2.0) 


2.6 
(1.8) 


2.7 
(1.6) 


4.8 
(1.8) 






Ease 

of Use-R 


5.5 
(1.2) 


6.4 
(0.7) 




5.9 
(0.9) 


5.9 
(1.2) 


5.5 
(1.1) 


5.7 
(1.2) 


6.3 
(0.9) 




5.9 
(0.8) 


5.4 
(1.2) 


5.0 
(1.7) 


6.9 
(0.3) 


5.1 
(0.9) 


4.3 
(1.6) 


5.8 
(1.1) 






Ease 

of Use-C 








5.7 
(0.9) 


5.8 
(1.1) 


5.1 
(1.6) 


6.2 
(1.1) 


6.2 
(0.9) 




6.5 
(0.5) 


5.8 
(1.2) 


4.9 
(1.2) 


6.3 
(0.9) 


5.1 
(1.1) 


5.1 
(1.4) 


5.6 
(1.0) 






Amount of 

Training 

Received-R 


3.3 
(1.3) 


4.5 
(1.2) 




3.5 
(1.5) 


4.4 
(1.4) 


3.9 
(1.4) 


4.2 
(1.3) 


4.2 
(1.3) 






3.0 
(1.5) 




4.4 
(1.1) 


2.8 
(1.7) 




4.8 
(1.8) 






Amount of 

Training 

Received-C 








3.2 
(1.9) 


4.2 
(0.9) 


4.9 
(1.7) 


4.8 
(1.5) 


4.1 
(1.7) 






3.3 
(1.4) 




3.2 
(1.9) 


3.0 
(2.0) 




3.3 
(1.8) 




Adequacy of 

Training 

Received-R 




















5.3 




5.5 
(1.3) 






4.8 
(1.8) 








Adequacy of 

Training 

Received-C 




















5.6 




5.4 

\ 1 .V) 






5.2 
(1.5) 








Training 
Value-R 


4.8 
(1.4) 


5.3 
(1.1) 




5.2 
(1.4) 


5.2 
(1.2) 


5.0 
(1.4) 


6.5 
(0.8) 


6.0 
(0.8) 




6.0 
(I.O) 


4.6 
(1.2) 


5.6 
(1.1) 


6.4 
(1.0) 


4.6 
(1.3) 


4.1 
(1.6) 


b.6 
(1.0) 




I 


Training 
Value-C 








5.0 
(1.0) 


4.8 
(1.6) 


5.4 
(1.7) 


5.7 

i 1 A) 


6.1 




4.5 
(2.2) 


4.4 
(1.2) 


5.1 
(1.6) 


5.1 
(1.4) 


4.4 

(LS) 


4.0 
(1.5) 


4.9 
(0.9) 




'j 


Potential 

Training 

Value-R 


5.0 
11.7) 


5.5 
(0.8) 


4.6 
(2.2) 


5.3 
(1.3) 


5.5 
(1.3) 


5.2 
(1.0) 


5.7 
(1.5) 


6.5 
(0.7) 


4.0 
(2.0) 


6.4 
(0.5) 


5.0 
(1.4) 


4.6 
(2.1) 


5.0 
(1.8) 


4.2 
(1.8) 


4.8 
(1.5) 


5.3 
(1.3) 


3.b 
(2.2) 


?; 


Potential 

Training 

Yalue-C 








5.1 
(1.5) 


5.4 
(1.6) 


5.3 
(1.9) 


5.8 
(1.3) 


6.0 
(1.2) 


4.1 
(2.5) 


5.5 
(2.1) 


5.2 
(0.9) 


5.4 
(1.5) 


4.1 
(1.6) 


5.1 
(1.3) 


4.6 
(1.4) 


5.5 
(1.2) 


2.7 
(1.5) 


i 
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maintaining a relatively realistic simulated environment. In contrast, on 
the A-10 ATD and the B-52 WST, it was more time-consuming to implement a 
partial than a total system freeze, and partial freeze was used 
significantly less often on these devices. 

Comparisons across MAJCOMs and levels of training. The utility and 
utilization of partial freeze did not differ icross the MAJCOMs. On one 
device, the F-15 simulator, there were clear differences across the two 
levels of training: F-15 RTU Sis used partial freeze more often, found it 
easier to use,> received more training in its use, and rated it higher in 
training value than did F-15 CTU Sis. However, the CTU Sis assigned 
higher potential training value ratings to total system freeze than did 
the RTU Sis. On the C-5A/C-141, partial .freeze was also used more often 
by RTU than by CTU Sis. 

Monitoring Features 

Parameters and Procedures Monitoring (Tables 17 and 18) 

Incidence. Parameters and procedures monitoring were surveyed on all 
devices except TAC's non-EW ATDs. Parameters monitoring was available for 
use oh the T-5, C-130, CH-3/HH-53, FB-lllA, T-4, B-52 WST, and A-10 (EW) 
ATDs. Procedures monitoring was available on the same devices, with the 
exception of the CH-3/HH-53 ATU. 

Utility and utilization . The utility and utilization of these 
features was very high. Over 90% of the means were in at least the 
moderate range of the rating scales. The ratings tended to be highest for 
those devices that required Sis to monitor performance from a remote 
console (e.g., FB-lllA, T-4), whereas the ratings were significantly lower 
for ATDs in which the instructor console was located in the simulation 
chamber with the student (e.g., C-130, CH-3/HH-53). Under these 
circumstances, the majority of Sis preferred to monitor student 
performance "over-the-shoulder" by looking at the instruments and switches 
directly. One exception to this trend was the A-10 ATD (EW), for which 
parameters and procedures monitoring received low ratings despite the 
remote location of these features. 

Comparisons across MAJCOMs and levels of training . The utility and 
utilization of parameters and procedures monitoring did not differ 
significantly across the MAJCOMs. There were significant differences 
across the two levels of training on only one device, the A-10 ATD (EW). 
The A-10 (EW) RTU Sis used parameters and procedures monitoring more often 
than did the A-10 (EW) CTU Sis. In addition, the A-10 (EW) RTU Sis rated 
parameters monitoring higher in training value than did the A-10 (EW) CTU 
Sis. 

Feedback Features 

Record/Playback (Table 19) 

Incidence , Record/playback was surveyed at every ATU site. It was 
available for use on all devices except the T-5, C-5/C-141, T-4, 
F-4E/F-4G, and F-15 ATDs, and the E-3A flight simulator. 
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Table 17 . Parameters Monitoring 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



CO 



Frequency 
of Use-R 

Frequency 
of Use-C 

Ease 

of Use-R 
Euse 

of Use-C 

Amount of 

Training 

Received-R 

Amount of 

Training 

Received-C 

Adequacy of 
Trai ni ng 
Recei ved-R 

Adequacy of 

Training 

Received-C 

Training 
Value-R 

Training 
Value-C 

Potential 

Training 

Value-R 

Potential 

Training 

Value-C 
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T-50 T-51 



T-5 
(EW) 



C-S/ C.130 CH-3/ 
C-141 



SAC 



FB-lllA FB-lllA T.4 B-52 WST 



TAC 



4.6 
(2.2) 



5.5 



6.0 
(K9) 



6.2 
(0.8) 



6.2 
(1.2) 









{ V\J\ 

\ tm) 










0*0 


o*o 


C Q 
0*O 


0* b 


(1.7) 


(2.2) 


(0*4) 


(0.6) 


(0.8) 


(0.9) 


3.2 


6.0 


6.7 


5.6 


6.7 


5.8 


(1 9) 








\ U.o J 




S.3 


5.8 


4.8 


5.0 




4.4 


(1.3) 


(1.4) 


(1.6) 


(l!3) 


(0.8) 


(l!7) 


3.6 


5.5 


4.8 


4.1 


6 0 


4.3 


(1.7) 


(1.3) 


(1.51 


(1.3) 


(l!3) 


(l!4) 


4.3 


4.0 


4.3 


5.3 






(1.2) 


(1.5) 


(1.8) 


(1.1) 






4.1 


4.8 


5.4 


5.6 






(1.0) 


(1.7) 


(1.4) 


(1.2) 














5.8 


5.6 










(1.1) 


(1.2) 



F-4E/ F-4G F-15 A-10 A-10 E-3A E-3A 
F-4G (EW) (EW) FS MS 













5.1 


5.4 












(0.7) 


(1.1) 


5.6 


4.7 


5.5 


6.5 


6.2 


6,0 


4.5 


(2.2) 


(1.5) 


(1.4) 


(0.8) 


(1.1) 


(1.3) 


(1.5) 




4.1 


5.5 


6.1 


G.5 


6.5 


6.2 




(1.9) 


(1.4) 


(1.0) 


(0.9) 


(0.8) 


(1.3) 


6.0 


4.7 5.2 


6.1 


6.8 


6.5 


5.6 


5.2 


(1.5) 


(1.5) (1.1) 


(1.4) 


(0.4) 


(0.5) 


(2.2) 


(2.2) 



4.8 4.7 5.5 6.4 6.7 5.5 6.3 

(1.6) (1.5) (1.7) (0.8) (0.6) (2.3) (1.6) 



(1 



.2 

.2) 



5.0 
(1.7) 



5.9 
(1.7) 

2.2 
(2.7) 

4.8 
(1.8) 

6.0 
(0.0) 



5.5 
(1.3) 



5.2 
(2.1) 

4.7 
(1.9) 

2.5 
(2.4) 



5.4 
(1.8) 



4.1 
(2.0) 
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Table 18 , Procedures Monitoring 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC 



MAC 



SAC 



TAC 



T-50 T-51 T-5 



C-5/ C-130 CH-3/ FB-lllA FB-lllA T-4 B-52 WST 



00 
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F-4E/ F-4G F-15 A-10 A-10 E-3A E-3A 

MS 







(EW) 


C-141 




HH-53 




(EW) 


(EW) 


(EW) 


F-4G (EW) 


(EW) FS 


Frequency 
of Use-R 




6.5 
(1.4) 




3.0 
(1.9) 




6.8 
(0.4) 


6.8 
(C.6) 


6.6 
(0.9) 


5.0 
(3.1) 




4.2 
(2.7) 


Frequency 
of Use-C 








2.8 
(1.4) 




6.6 
(1.1) 


6.5 
(0.9) 


6.6 
(0.9) 


4.7 
(2.9) 




1.3 
(0.5) 


Ease 

of Use-R 




6.1 
(0.8) 




4.8 
(1.3) 




5.0 
(1.5) 


4.6 
(1.7) 


4.9 
(1.4) 


5.2 
(1.6) 




4.8 
(1.8) 


Ease 

of Use-C 








4.4 

(1.4) 




4.5 
(1.6) 


3.6 
(1.6) 


5.2 
(1.1) 


3,2 
(1.8) 




4.0 
(1.6) 


Amount of 

Training 

Received-R 




• 




3.6 

(1.5) 




4.5 

(2.0) 


5.2 
(1.1) 










Amount of 

Training 

Received-C 








4.2 
(1.0) 




5.7 
(1.3) 


5.6 
(1.2) 










Adequacy of 
Training 
Recei ved-R 




6.4 
(1.1) 












5.8 
(1.2) 


4.8 
(2.1) 




5.9 
(1.2) 


Adequacy of 

Training 

Received-C 
















5.8 
(1.1) 


4.6 
(2.0) 




£.1 

IC 8) 


Training 
Value-R 




6.1 
(1.8) 




4.2 

(1.7) 




6.5 

(0.8) 


6.2 

(1.1) 


6.0 
(1.5) 


3.0 
(1.7) 




4.3 
(2.2) 


Training 
Value-C 








3.7 
(1.8) 




6.2 
(0.9) 


6.7 

(0.8) 


6.3 
(1.2) 


4.3 
(2.4) 




2.7 
(2.0) 


Potential 

Training 

Value-R 


4.3 
(2.0) 


5.0 6.5 
(1.5) (1.0) 


4.8 
(1.6) 


4.S 
(1.2) 


4.2 
(1.9) 


6.8 
(0.4) 


6.6 
(0.5) 


6.0 
(1.4) 


4.7 
(2.2) 


4.8 

(!.4) 


4.9 
(1.9) 


Potential 

Training 

Value-C 






5.0 
(1.6) 


4.4 
(1.6) 


3.9 
(2.2) 


6.7 

(o.e) 


6.9 

(0.3) 


6.0 
(2.1) 


6.2 
(1.6) 


4.7 
(1.9) 


4.1 
(1.7) 
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Table 19 . Record/Pl ayback 

Mean Ratings (and Standard Deviations) of the Frequency of Use» 
Ease of llse» Amount of Training Received^ Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 







ATC 






MAC 






SAC 










TAC 










T-50 


T-51 


T-5 
(EW) 


C-5/ 
C-1 41 


C-130 


CH-3/ 
HH-53 


FB-111A FB-lllA 
(EW) 


T-4 

(EW) 


B-52 WST F-4E/ 
(EW) F-4G 


F-4G 
(EW) 


F-15 


A-10 


A-10 
(EW) 


E-3A 
FS 


E-3A 
MS 


Frequency 
of Use-R 


3.4 
(1.3) 


3.9 
(1.3) 






1.9 
(1.2) 


2.2 
(1.2) 


3.0 
(1.7) 


2.3 
(0.8) 




1.2 
(0.4) 






1.6 
(0.7) 


1.1 
(0.3) 




1.2 
(0.6) 


Frequency 
of Use-C 










2.0 
(0.7) 


3.8 
(1.4) 


1.7 
(0.9) 


1.8 
(0.7) 




1.0 
(0.0) 






1.3 
(0.7) 


2.2 
(1.0) 




1.3 
(0.8) 


Ease 

of Use-R 


4.9 

(1.0) 


5.4 
(1.1) 






4.1 

(0.8) 


4.4 
(1.5) 


3.6 
(1.5) 


3.9 
(1.4) 




2.8 
(1.1) 






3.5 
(1.3) 


2.3 
(1.0) 




2.3 
(1.4) 


Ease 
of Use-C 










4.3 
(1.6) 


5.2 
(1.0) 


3.3 
(1.7) 


3.4 
(1.1) 




3.2 
(1.5) 






4.7 
(0.6) 


4.5 
(1.1) 




1.: 

(0.7) 


Amount of 

Training 

Received-R 


3.5 
(1.2) 


4.6 
(0.9) 






2.8 
(1.4) 


2.1 
(0.9) 


3.5 
(1.5) 


2.5 
(1.3) 










2.3 
(1.1) 






1.1 

(0.3) 


Amount of 

Training 

Received-C 










3.5 
(1.2) 


4.5 
(1.2) 


2.3 
(1.3) 


2.1 
(1.1) 










1.7 
(1.4) 






1.3 
(0.8) 


Adequacy of 

Training 

Received-R 




















3.5 
(1.8) 








3.2 
(1.9) 






Adequacy of 

Training 

Received-C 










• 










3.1 
(2.0) 








5.0 
(1.5) 






Traini ng 
Value-R 


4.9 
(1.0) 


5.5 
(1.3) 






3.0 
(1.4) 


4.1 
(1.7) 


4.2 
(0.8) 


3.8 
(1.5) 




3.2 
(2.1) 






4.1 

(1.2) 


1.5 
(0.5) 




4.6 
(2.1) 


Training 
Value-C 










3.4 
(1.4) 


5.0 
(1.8) 


4.0 

(1.7) 


3.3 
(1.4) 




1.7 
(1.2) 






3.4 
(1.3) 


3.7 
(1.4) 




2.3 
(0.6) 


Potential 

Training 

Value-R 


5.2 
(1.3) 


5.9 
(1.2) 


5.3 
(1.4) 


4.5 
(1.6) 


4.0 
(1.6) 


4.6 
(1.8) 


5.3 
(1.0) 


5.2 
(1.5) 


4.6 
(2.0) 


5.8 4.5 
(0.7) (1.9) 


5.5 
(1.5) 


5.4 
(1.1) 


4.7 
(1.2) 


3.7 
(1.7) 


5.6 
(1.1) 


6.0 
(1.6) 


Potential 

Training 

Value-C 








4.8 
(1.7) 


4.3 
(1.4) 


5.8 
(1.6) 


4.4 
(1.7) 


4.4 

(1.7) 


6.1 
(1.0) 


4.7 5.5 
(1.8) (0.9) 


5.3 
(1.6) 


5.5 
(1.3) 


5.3 
(0.9) 


4.4 

(1.5) 


1.5 
(0.7) 


4.8 
(1.6) 



66 



Utnicy and utilization . Record/playback was among the lowest rated 
of all instructional features. At two sites (viz., T-50, T-51), it was 
fairly easy to use and the ratings were in the moderate range. At each of 
the remaining sites, however, record/playback was rated difficult and 
time-consuming to implement, and operationally unreliable. Over 70% of 
the B-52 WST, A-10, A-10 (EW), and E-3A mission simulator Sis reported 
never having used the feature. 

Comparisons across MAJCOMs and levels of training . The utility and 
utilization of record/playback did not differ across the MAJCOMs. There 
was only one significant comparison across the two levels of training: 
E-3A flight simulator RTU Sis rated record/playback higher in potential 
training value than did E-3A flight simulator CTU Sis. 

Automated Performance Feedback (Table 20) 

Incidence . Automated performance feedback was surveyed at each 
non-EW TAC ATD site. It was available for use only on the F-4E ATD and 
the E-3A mission simulator. 

Utility and utilization . Automated performance feedback was 
apparently easy to use on the E-3A mission simulator, and its frequency of 
use was in the low to moderate range of the scale. In contrast, the 
feature was difficult and time-consuming to implement on the F-4E ATD, and 
its frequency of use was at the low end of the scale. In fact, the 
majority of F-4E Sis reported never having used the feature, preferring 
instead to "freeze" the mission and give verbal feedback; and over 30% 
were unaware of its availability. 

Comparisons across MAJCOMs and levels of training . Comparisons 
across the MAdCUMs were precluded since data were collected only from TAC 
ATDs. On four of the five devices surveyed, the RTU Sis' mean rating of 
the potential training value was greater than that of the CTU Sis. The 
difference was significant only for the E-3A flight simulator, however. 

Hard Copy (Table 21) 

Incidence . Hard copy was surveyed at every ATD site. It was 
available for use on all devices except the T-50, T-51, C-5/C-141, 
CH-3/HH-53, T-4, and A-10 (EW) ATDs. 

Utility and utilization . The frequency of use of hard copy was 
uniformly low, wun mean ratings all in the "rarely" to "occasionally" 
range of the scale. The only successful implementation of this feature 
appeared to be on the T-5 ATD, where utility and ease of use were very 
high. Yet, it was used only once every two to four missions, on the 
average. Hard copy was one of the most problematic features surveyed. 
For example, on the C-130 ATD, it was said to be seldom operational and 
time-consuming to implement. On the B-52 WST, it was said to yield output 
that was difficult to interpret. And on the F-4G (EW) ATD, it was called 
"unreliable" by 28% of the Sis and was either never operated or presumed 
unavailable by 53% of the Sis. 
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Table 20, Automated Performance Feedback 



Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC MAC SAC JAC 



T-50 T-51 T-5 C-5/ C-130 CH-3/ FB-lllA FB-lllA T-4 B-52 WST F-4E/ F-4G F-IS A-10 A-10 E-3A E-3A 





\ uH ) 


l#— 1 H 1 


nn-DO 


f ru \ 1 ru \ 


( tW) 


r-4G 


(EW) 




(EW) FS 


MS 


Frequency 
of Use-R 












1.5 
(0.8) 








3.4 
(2.6) 


Frequency 
of Use-C 












1.7 
(0.7) 








2.7 
(2.6) 


Pa CO 

Of Use-R 












2.1 
(0.9) 








6.3 
(0.8) 


Ease 

rk'f ilea C 

Or USe*L 












3.0 
(0.0) 








6.0 
(1.4) 


Amount of 
Training 
Kecei veo-K 




- 








1.6 
(1.6) 








2.4 
(2.1) 


Amount of 
Trai nina 
Recei ved-C 












1 .J 

(0.6) 








2.2 
(1.8) 


Adeauacv of 
Training 
Recei ved-R 






















Ad^oLiarv of 
Training 
Recei ved-C 






















Training 
Value-R 












3.1 
fl.O) 








4.1 
(2.3) 


Training 
Value-C 












3.3 

(2.3) 








5.0 
(0.0) 


Potential 

Training 

Value-R 












4.4 

(1.7) 


4.3 
(1.8) 


3.8 
(1.7) 


5.0 
(1.4) 


5.3 
(1.8) 


Potential 

Training 

Value-C 












4.3 
(0.6) 


3.3 
(1.7) 


3.8 
(1.7) 


1.5 
(0.7) 


3.8 
(1.9) 
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Table 2K Hard Copy 



Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received, Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



ATC MAC SAC ^TAC 



T-50 T-51 T-5 C-5/ C-130 CH-3/ FB-1 IIA FB-1 1 lA T-4 B-52 WST F-4E/ F-4G F-15 A-10 A-10 E-3A E-3A 







(EW) 


C-141 




HH-53 




(EW) 


(EW) 


(EW) 


F-4G 


(EW) 






(EW) 


FS 


US 


Frequency 
of Use-R 




2.9 
(1.7) 




2.6 
(1.2) 




1.2 
(0.4) 


1.5 
(0.7) 




2.2 

(0.7) 


1.8 
(0.8) 


2.3 
(1.8) 


1.2 
(0.5) 


2.6 
(1.6) 




1.0 
(0.0) 


2.6 
(1.2) 


Frequency 
of Use-C 








2.1 
(0.8) 




1.3 
(0.6) 


1.3 
(0.7) 




2.0 
(1.5) 


1.8 
(1.3) 


2.0 
(1.5) 


2.1 
(1.5) 


1.6 
(1.2) 




1.0 
(0.0) 


2.8 
(1.8) 


Ease 

of Use-R 




6.2 
(1.0) 




4.7 
(1.4) 




3.0 
(2.0) 


2.0 
(0.8) 




4.4 

(1.8) 


3.3 
(1.6) 


4.2 
(1.6) 


4.8 
(2.1) 


4.6 
(1.5) 




2.0 
(1.4) 


5.6 
(1.0) 


Ease 

of Use-C 








4.8 
(l.fa) 




2.4 
(1.9) 


3.0 
(1.8) 




4.5 
(0.8) 


3.5 
(1.8) 


4.1 
(2.1) 


4.1 
(1.4) 


5.2 
(1.0) 




- 


5.6 
(0.9) 


Amount of 

Training 

Received-R 








3.4 

(1.7) 




1.2 

(0.4) 


1.4 

(0.7) 






2.0 
(l.o) 




1.3 
(0.7) 


3.3 
(1.3) 




1.0 

(0.0) 


3.3 
(1.5) 


Amount of 

Training 

Received-C 








3.8 

(1.0) 




1.4 

(0.7) 


1.4 

(0.7) 






1.8 

(1.3)- 




2.5 
(1.1) 


^:.8 

(1.8) 




1.0 
(0.0) 


4.0 
(0.6) 


Adequacy of 

Training 

Received-R 




6.0 

(1.2) 














4.8 
(1.9) 




4.4 
(1.6) 












/''dequacy of 

Training 

Received-C 


















5.2 
(1.3) 




4.6 
(2.1) 












Training 
Value-R 




6.1 

(1.7) 




3.1 

(1.3) 




1.7 

(1.2) 


2.2 

(0.4) 




3.1 
(1.5) 


4.0 
(1.5) 


3.6 
(1.5) 


3.6 
(1.5) 


4.0 
(1.6) 




1.0 
(0.0) 


3.7 
(1.5) 


Training 
Value-C 








3.2 
(1.6) 




2.7 
(1.3) 


3.1 
(1.4) 




2.5 
(1.5) 


4.7 
(0.3) 


3.7 
(1.7) 


3.7 
(l.S) 


3.0 
(1.7) 






3.2 

;o.8> 


Potential 

Training 

Value-R 


4.5 
(2.0) 


4.5 6.6 
(1.3) (1.0) 


3.7 
(1.6) 


4. , 

(1.5) 


3.9 

(1.8) 


3.5 
(2.1) 


4.5 
(1.9) 


3.2 
(1.9) 


5.2 
(1.6) 


4.7 
(1.2) 


4.6 
(1.1) 


4.5 
(1.6) 


4,4 
(1.5) 


4.1 

;i.8) 


4.0 
(1.9) 


4.6 
(1.6) 


Potential 

Training 

Value-C 






3.8 
(1.7) 


3.9 
(1.8) 


4.1 
(2.3) 


4.1 
(1.6) 


4.1 
(1.6) 


4.1 
(2.2) 


4,3 
(2.3) 


4.4 
(1.5) 


b.2 
(1.3) 


4.2 

(i.a) 


3.5 
(1.6) 


4.3 
(1.6) 


3.3 
(2.1) 


2.8 
(1.1) 



Comparison across MAJCOMs and levels of training . The utility and 
utilization of the hard copy feature did not differ across the MAJCOMs. 
There was only one significant comparison across the two levels of 
training: F-15 CTU Sis received more training in the use of hard copy than 
did T-15 RTU Sis. 

Performance Scoring (Table 22) 

Incidence . Performance scoring was surveyed at every EW ATD site. 
The feature was available for use on the T-5, B-52 WST, and A-10 devices. 

Ut i 1 i ty an d u t i 1 i za t i on . Despite reports of "unreliability" on the 
B-52 and AH 0 AIDS , performance scoring generally received higher utility 
and utilization ratings than the other feedback features (e.g., record/ 
playback and hard copy). On the T-5 ATD, performance scoring was used 
between two and four times each mission, on the average. A-10 RTU Sis 
reported using the feature even more frequently, between five and seven 
times each mission, on the average. Performance scoring was rated as 
"moderately" (B-52 WST) to "extremely" (T-5) useful. 

Comparisons across MAJCOMs and levels of training . Comparisons 
across the I^JCOMs were precluded since data on this feature were 
collected from relatively few ATDs. There was only one significant 
comparison across the two levels of training: A-10 RTU Sis used 
performance scoring more frequently than did A-10 CTU Sis (an average of 
five to seven times a mission versus once every two to four missions, 
respectively). 

Amount and Adequacy of Training Received by Simulator Instructors 

Sis generally received a greater amount of training in the use of 
those features that were also rated higher in utility and utilization. 
This was indicated by the significant positive Intercorrelations between 
the ratings of each AIF on tne five survey questions. There were clear 
differences in the amount of training received across the MAJCOMs. TAC 
mean ratings (non-electronic warfare) were lower than those of the other 
MAJCOMs for over 81% of the features surveyed. Moreover, TAC Sis tended 
to characterize their training as "informal." In contrast, ATC, MAC, and 
SAC Sis (non-EW) tended to characterize their training as "formal." 

Observations concerning the adequacy of training received by the Sis 
must be considered tentative, since the relevant data were collected only 
during the EW phase of the project. Nevertheless, the high ratings 
suggest that the Sis from ATC, MAC, SAC, and TAC would each characterize 
their training as adequate. 

Some additional data of interest were collected during the EW phase 
of the project. These related to the relative impact of "initial" and 
"refresher" training on AIF utility and utilization. The amount of formal 
initial training appeared to relate positively to the magnitude of the 
ratings. For example, T-4 and T-5 Sis received a greater amount of formal 
training than did other Sis (classroom instruction accounted for 54% and 
28% of initial training, respectively) > and the T-4 and T-5 ATDs were 
probably the most favorably evaluated devices. The impact of refresher 
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Table 22 , Performance Scoring 

Mean Ratings (and Standard Deviations) of the Frequency of Use, 
Ease of Use, Amount of Training Received^ Adequacy of Training Received, 
Training Value, and Potential Training Value by RTU (R) and CTU (C) Simulator Instructors 



4> 



ATC MAC SAC TAC 

T-50 T-51 T-5 C-5/ C-130 CH-3/ FB-lllA FB-lllA T-4 B-52 WST F-4E/ F-4G F-15 A-10 A-10 E-3A E-3A 
(EW) C-Hl HH-53 (EW) (EW) (EW) F-4G (EW) (EW) FS MS 



Frequvincy 5.3 2 9 6 1 

of Use-R (2.1) (1.9) (K?) 

Frequency 4,2 3 3 

of Use-C (2i6) (2.'2) 

Ease 6.4 5.1 4 6 

of Use-R (1.0) (1.6) (1.7) 

Ease 3.8 4 3 

of Use-C (1.9) (^6) 



Amount of 
5* Training 
Received-R 



Anount of 

Training 

Received-C 

Adequacy of 

Training 6.6 5.O 5 s 

Received-R (0.7) (1.8) (1^2) 

Adequacy of 

Training 5.2 45 

Received-C (2.0) (l]?) 

Training 6.2 3.8 3 7 

Value-R (1.5) (1.8) (2;o) 

Training 3.O 3 3 

Value-C ^ (2.3) {]\q) 

Potential 

T**aining 6.3 4.2 5.8 5.0 5 9 

Yalue-R (1.2) (1.9) (1.3) (2.0) (o!9) 

Potential 

l'»*aining 4.4 5.3 5^2 4 6 

^alue-C (2.1) (2.3) (1.8) (2.0) 
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training was less clear. For example, T-4 and A-10 (EW) Sis received a 
greater amount of refresher training than did other Sis (42% of T-4 Sis 
and 45% of A-10 Sis received refresher training at least once within the 
immediately preceding year), but the A-10 ratings tended to be much lower 
than those of the T-4. 



IV. DISCUSSION 
General Trends 
Interrelations Among the Ratings 

The results indicated that an AIF was used to the extent that: Sis 
were trained in its use, it was easy to use, and it had apparent training 
value. However, training value was clearly the most significant predictor 
at almost every ATD site surveyed. What can be concluded from these 
facts? Unfortunately, correlational findings do not logically imply 
causality; they merely reflect the presence of a relationship among 
variables. In this case, however, it seems reasonable to assume that 
particular AIFs were used, more frequently because they were more useful. 
Indeed, assuming that the training value of an AIF did not affect its use 
is clearly implausible. 

How can the fact be explained that the remaining variables (viz., 
ease of use, amount/adequacy of training received) did not account for 
much of the variability in frequency of use? This fact suggests that Sis 
would not avoid using a particular feature, even if it were complicated to 
use, as long as they believed that it would help accomplish mission 
objectives. 

Overview of the Rating Data 

There were two general trends revealed in the overview of the ratiny 
data (Figures 1 through 6). The first liad to do with differences in 
utility and utilization among the various types of features. Thus, 
training management, variation of task difficulty/fidelity, and monitoring 
features tended to receive the highest ratings, whereas briefing and 
feedback features tended to receive the lowest ratings. The fact that 
this trend was observed at every ATD site suggests that, regardless of 
differences in particular mission requirements. Sis adopted a common 
strategy for the use of instructional features. Apparently the strategy 
was one in which the use of AIFs was concentrated during the actual 
mission performance (e.g., for training management). Although certain 
AIFs could be used outside the mission context (e.g., for briefing), they 
were used less often. There were two notable exceptions to this trend. 
Automated adaptive training, a training management feature, received 
extremely low utility and utilization ratings, whereas performance 
scoring, a feedback feature, generally received high ratings. These two 
AIFs will be discussed in greater detail below. 

The second general trend revea'ted in the overview of the data had to 
do with differences in the ratings of certain features across the two 



It 

^ levels of training. The differences between the RTU and CTU Sis' use of 
1^, AIFs appeared to reflect a critical distinction betvfeen the two types of 

b' missions. RTU missions tended to stress the acquisition of discrete 

r ' skills and procedures, whereas CTU missions tended to maintain the broader 

iv mission context. Thus, RTU Sis Were more likely to interrupt or otherwise 

alter the mission- in order to afford students sufficient opportunity for 
practice ;and feedback;, they more frequently invoked features such as 
freeze, reset, environmental, and parameters and procedures monitoring in 
order to accomplish this objective. In contrast, CTU Sis spent less time 
interrupting the mission in order to achieve a more continuous and 
realistic scenario. Programmed scenarios were ideally suited for this 
purpose, and they were more frequently invoked by CTU Sis. 

It is important to point out, however, that the differences described 
above were not evidenced at each ATD site. In fact, there were no 
significant differences between RTU and CTU "atings on over 4034 of the 
devices, excluding.ATC (viz., CH-3/HH-53, T-4, B-52 WST, F-4E/F-4G, F-4G 
EW, E-3A mission simulator). Even on devices where differences were 
observed (e.g., F-15 ATD), the majority of features were used similarly by 
RTU and CTU Sis. Thus, Sis were more similar in their use of AIFs than 
they were different. Both groups used the same fc;atures to one degree or 
another, and each ATD could readily accoiiriodate either type of training. 

Utility and Utilization Ratings 

Briefing Features 

The briefing features, recorded briefing and demonstration, tended to 
be among the least valued and least used features of those surveyed. The 
Sis* comments suggested that there were two major reasons for the low 
ratings. Firsts Sis generally preferred to brief students themselves. 
This was probably a more reasonable strategy since individual students* 
needs might differ, and a "generic" briefing might not be appropriate for 
all students. Second, the low ratings undoubtedly reflected operational 
difficulties and limitations associated, particularly, with 
demonstration. The most commonly reported problems were "time-consuming 
implementation" and "unreliability." This would seem unfortunate, since, 
according to social learning theory, "a large amount of human learning is 
done vicariously , through observing another person making the skilled 
responses... and then trying to imitate the response of the model" 
(Hilgard & Bower, 1975, pp. 599-505). Nevertheless, empirical evidence 
from the aircrew flight training literature suggests that the use of 
demonstration is no more effective for training than is simple performance 
feedback (Hughes et al., 1979). 

The potential training va^-e mean ratings were unremarkable. They 
were similar across the ATDs * jd were all in the moderate range of the 
scale. None of the devices surveyed had an instructor tutorial 
capability, and it is therefore difficult to discuss the u':ility and 
utilization of this feature. However, the few Sis that commented on 
instructor tutorial apparently preferred "hands-on" experience and/or 
"face-to-face" tutorials for learning the operation of the instructor 
console. 
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Training Management Features 

Freeze and reset were among the highest rated of all features 
suirveyed* This was true for virtually every device that was surveyed. 
The ability to temporarily suspend a mission in order to instruct a 
student, and then, allow the mission to continue, appeared to be essential 
for effective training management, especially for RTU training, in which 
the acquisition of discrete skills and procedures was stressed. Yet, the 
empirical evidence for tha training effectiveness of these features is 
less clear. Bailey^ Hughes, , and Jones (1980) successfully used freeze and 
reset in their ap'plicatipn of the behavioral "backward chaining" paradigm 
to air-torsurface weapon delivery training. On the other hand, Hughes et 
al. (1982) Were unable to significantly Improve carrier glideslope 
tracking performance by using freeze as opposed to a more conventional "no 
freeze" training approach. Their approach was to freeze the simulator 
whenever a glideslope error was detected. The simulator was then returned 
to the previous position, with appropriately configured angle-of-attack 
and airspeed, and the student was alloweu to try again. 

Crash-kill override was typically "left on" for most training 
applications in order to avoid reinitializing the simulator following a 
"crash" or "kill" and-* thus, preserve instruction time. "Crashes" were 
allowed to occur more frequently on ATC trainers (viz., T-50, T-51). 
Several ATC Sis commented that entry-level pilots must be taught to be 
sensitive to dangerous conditions. Experiencing a simulated crash is 
apparently one way to accomplish this objective. 

Automated adaptive training received the lowest utility and 
utilization ratings of the training management features; however, it is 
difficult to draw, any firm conclusions since the feature was operational 
on only one ATD, the F-4E/F-4G. Notwithstanding the low ratings, there is 
considerable evidence that automated adaptive training has training 
utility (Brown, Waag, & Eddowes, 1975; Charles, Johnson, & Swink, 1971, 
1973; Charles, Willard, & Healey, 1975; Feuge, Charles, & Miller, 1973). 
According to Brown et al. (1975), the major difficulties with automated 
iJdaptive training are limited training scenarios, the high cost of 
software implementation, the lack of formal instructor training in the use 
of the feature, and the lack of training directives to implement such 
training. 

Programmed mission scenarios were used to "streamline" a training 
session by freeing a simulator instructor to perform other important 
duties, such as monitoring student performance and giving feedback. 
Programmed scenarios were especially useful for CTU missions, which tended 
to be interrupted less often than were RTU missions. However, as 
discussed earlier, there were certain operational limitations associated 
with the use of programmed mission scenarios. Thus, the programming of 
scenarios was tedious and difficult, and there were few available 
scenarios at most training sites. Moreover, those that were available 
were frequently characterized as "unreliable," "limited," or "outdated." 
Although manual mission control afforded the SI a greater degree of 
instructional flexibility, it required considerable time and effort. 



47 



77 



Variation of Task Difficulty/Fidelity Features 

Most emergencies can be safely dealt with only under simulated 
conditions, and the frequent Insertion of simulated malfunctions was 
characteristic of every training mission that was observed. However, the 
same problems that limited the use of programmed mission scenarios also 
limited the use of automated malfunction insertion at the majority of ATD 
sites. It was said that it was time-consuming to program and implement 
and unreliable, and did not always reflect mission profiles. Roughly one 
in five Sis preferred to insert malfunctions manually, despite the greater 
effort that was required. 

^ Of the two "fidelity" features that were surveyed, environmental and 
motion, environmental appeared to be the least problematic. The utility 
and utilization ratings of the environmental feature were uniformly 
favorable, except for the E-3A mission simulator, which had a very limited 
environmental capability. Platform motion cueing was more frequently used 
and was considered a more useful training feature than was environmental; 
however, at every ATD site, the motion system was difficult to maintain 
and was sometimes Inoperable. 

As described in the Results section, motion received higher utility 
ratings from non-fighter Sis (e.g., C-130, E-3A flight simulator) than 
from fighter Sis (e.g., F-15). This difference was probably due to the 
fact that platform motion cueing systems are not capable of high-fidelity 
simulation of fighter aircraft movement. Empirical evidence suggests that 
platform motion cueing is more useful in non-fighter applications. Thus, 
Ricard, Parrish, Ashworth, and Wells (1981) found that platform motion 
cueing was effective for helicopter hover simulation training, whereas 
Martin and Waag (1978a, 1978b) found that it was ineffective for-basic 
contact maneuvers and aerobatics. The general utility of motion was also 
questioned by Cyrus (1978), who concluded in his literature review that, 
for most tasks, the elimination of platform motion cues does not reduce 
training effectiveness. 

The final feature in this group, partial freeze, was most often used 
as a substitute for total system freeze; and both features shared high 
utility and utilization ratings for similar reasons. However, partial 
freeze also permitted Sis to vary the student's task load by selectively 
freezing particular aircraft parameters. Partial freeze may therefore 
offer certain advantages over total system freeze, since it can be used 
both to manage training and to vary task difficulty. 

Monitoring Features 

Parameters monitoring and procedures monitoring permitted an SI to 
monitor student performance during a simulated mission by means of 
alphanumeric and/or graphic CRT displays of performance data. Not 
surprisingly, parameters and procedures monitoring were among the highest 
rated features. 
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The mon 1 tor fng displays were usually viewed at a remote instructor's 
console. Alternate but less-sophisticated capabilities for monitoring 
student performance from a remote console included repeaters (replicas of 
flight instruments) and annunciators (indicators that are directly linked 
to aircraft controls and switches). On some devices (e.g., C-130 Afi), 
CH-3/HH-53 AID), the instructor console was located within the simulator 
cockpit. In these situations. Sis could monitor performance by means of 
displays or b:' directly viewing the student "over-the-shoulder." 
Interestingly, repeaters, annunciators, or "over-the-shoulder" were often 
the preferred means of monitoring student performance* This suggests that 
Sis, most of whom are experienced aircrew members themselves, find it 
easier to monitor parameters and procedures in ways similar to those used 
during actual flying. This also suggests that the utility of remote 
paraiP^tGrs and procedures monitoring displays will depend on the format in 
which performance information is presented; e.g., digital displays of 
round dial instrument readings will be unacceptable to most Sis. 

Feedback Features 

Record/playback and hard copy were available for use on the majority 
of ATDs surveyed. The ratings of these features were uniformly low, with 
the exception of the ATC devices (e.g., record/playback on the T-50 and 
T-51 ATDs, and hard copy on the T-5 ATD). The utility of the record/ 
playback and hard copy features was limited by operational problems that 
discouraged their use. Both features were difficult and time-consuming to 
implement and were operationally unreliable. Had these features 
functioned reliably and efficiently, they might have been used more 
often. Indeed, the giving of feedback was a normal instructional 
procedure at every ATD site, although such feedback was more often given 
verbally than by means of AIFs. 

The amount of data collected on automated peiformance feedback and 
performance scoring was too small to support any firm conclusions. 
Automated performance feedback was operational on only two devices, the 
F-4E ATD and the E-3A mission simulator. The majority of F-4E Sis had 
never used the feature because it was difficult and time-consuming to 
implement. The E-3A mission simulator Sis used it infrequently. The 
potential training value ratings suggested that automated performance 
feedback could be useful, especially for RTU training. One advantage of 
automated performance feedback is that it is relatively unobtrusive, since 
it merely presents an "error cue" while allowing the mission to continue. 
By using automated performance feedback during RTU missions, where errors 
are more frequent, the instruction time that is normally used for verbal 
feedback could be saved. 

In contrast to the other feedback features, performance scor^'ng 
received generally favorable utility and utilization ratings. There are 
two possible reasons for the high ratings. First, the feature was easy to 
use and was reliable.- Second, as with automated performance feedback, 
performance scoring saved instruction time by "summarizing" performance 
automatically. 
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The only experimental evidence of the utility of the feedback 
features concerns record/playback. Hughes, Hannon, and Jones (1979) found 
that the periodic use of a replay of student performance was more 
effective in reducing errors during subsequent performance than was the 
use of an instructor-recorded "demonstration." However, record/playback 
was no more effective than simple practice. 

Training Received by Simulator Instructors 

As described in a previous section of this report, there were 
considerable differences across ATD sites in the amount and type of 
training received by the Sis. TAC Sis apparently received less training 
than did ATC, MAC, or SAC Sis; and TAC training was more often 
characterized as "informal." Yet the electronic warfare results indicated 
that TAC, ATC, MAC, and SAC Sis each rated their training as adequate, 
despite the stated differences in amount and type. This suggests that 
there may not be a "best" way to train Sis, but an important question 
needs to be answered before any firm conclusions can be drawn: What, 
precisely, are the appropriate criteria for "adequate" training? These 
will need to be empirically determined. 



V. RECOMMENUATIONS 

Improve the Training of Sis 

It io clear from this survey that the existing AIF capabilities of 
Air Force ATDs have not been fully explored. This is partly due to 
operational problems that have precluded the use of certain features, but 
it is also likely due to insufficient training of Sis. 

The extensive AIF capabilities of modern ATDs provide an inherently 
flexible and dynamic training environment. Many different AIFs can be 
implemented singly or in combination at various points during a simulated 
mission. What should Sis be taught about implementing AIFs? Whatever 
form the training of Sis takes (the results of the survey suggest that 
both formal and informal training may be effective). Sis must be taught 
not only how to use the available AIFs but also how to use them 
effectively (i.e., in ways that will maximize the acquisition and 
retention of aircrew skills). 

Specify Guidelines for Using AIFs Effectively 

Guidelines for using AIFs effectively need to be expressed in 
operational terms. It is not sufficient for Sis to know how to use a 
feature; they must also know when to use it. That is. Sis must know when 
to use a feature during a mission in order to maximize student performance. 

Guidelines for effective AIF use still need to be specified, 
however. Such guidelines cannot be derived from surveys. They require 
the conducting of experiments that compare the effects of implementing or 
not implementing a particular AIF on various performance criteria (e.g., 
deviation from glideslope or accuracy of weapon delivery). The few 
experiments of this type that have been conducted were mentioned in 
previous sections of this report. Many more are needed. 
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Improve the Operability of Advanced Instructional Features (AIFs) 

If the instructional capability of ATDs is to be fully realized, AIFs 
will need to be made more reliable and user-friendly. This survey 
revealed that there were operational problems with one or more features at 
every ATD site* A feature cannot be used effectively if it is difficult, 
time«consuming, or simply irpossible to implement. ATD design 
requirements must ensure that the full range of instructional capabilities 
can be utilized and maintained. 



81 



REFERENCES 



Adams, J. A. (1972). Research ind the future of engineering psychology. "* 
American Psychologist . 27, 615-622. 

Bailey, J.S., & Hughes, R.G. (1980). Applied behavior analysis in flying 
training research (AFHRL-TR-79-38, AU-AU8I /50). Winiams AFb, AZ: 
Hying Training Division, Air Force Human Resources Laboratory. 

Bailey, J.S., Hughes, R.G., & Jones, W.E. (1980) . Application of backward 
chaining to air-to-surface weapons delivery training 
lAFHRL-TR-79-b3, AD-A085 610). VJilliams APB, AZ: pfying Training 
Division, Air Force Human Resources' Laboratory. 

Brown, J.E., Waag, W.L., & Eddowes, E.E. (1975). USAF evaluation of an 
automated adaptive flight training system (A}-'HkL-TR-7S-55, AD-A018 
612). Williams AFB, AZ: Flying Tra1n1ng"Division, Air Force Human 
Resources Laboratory. 

Caro, P.W. (1973). Aircraft simulators and pilot training. Human Factors, 
15, 502-509. 

Caro, P.W., Pohlman, L.D., & Isley, R.N. (1979). Development of simulator 
instructional feature design guides (Sevine-TR-79-12, AD-A084 428). 
Pensacola, FL: Seville Research Corp. 

Charles, J. P., Johnson, R.M., & Swink, J.R. (1971). Automated flight 
training (AFT): instrument flight maneuver s 
TWrRAE()UlPCEN-71-C-O205-l, AD-759 366). Orlando, FL-. Naval 
Training Equipment Center. 

Charles, J. P., Johnson, R.M., & Swink, J.R. (1973). Automated flight 
training (AFT); GCI/CIC air attack (NAVTRAEQUIPCLN-72-C-0108-1 , 
AD-772 593). Orlando, FL: Naval Training Equipment Center. 

Charles, J. P., Willard, G., & Kealey, G. (1975). Instructor pi lot's role in 

simulator training (NAVTRAEQUIPCEN-75-C-0093- 1 , AD-A023 546) . 

Orlando, FL: Naval Training Equipment Center. 

Cyrus, M.L. (1978). Motion systems role in flight simulators for flying 
training (AFHRL-1R-/8-39, AD-A059 744). Williams Af-'B, Az: Flying 
Training Division, Air Force Human Resources Laboratory. 

Easter, A.W., Kryway, J.T., Olson, W.R., Peters, S.M., Slemon, G.K., & 
Obermayer, R.W. (19C6a). Development of instructor support feature 
guidelines {AFHRL-TR-85-57(I) , AD-A168 308). Williams AFB, AZ: 
Operations Training Division, Air Force Human Resources Laboratory. 

Easter, A.W. , Kryway, J.T., Olson, W.R., Peters, S.M., Slemon, G.K., & 
Obermayer, R.W. (1986b). Instructor support feature gui delines 
(AFHRL-TR-85-57(II), AD-AI68 285). Williams AFB, AZ: Operations 
Training Division, Air Force Human Resources Laboratory. 




52 



82 



Faconti, V,, & Epps, R, (1975). Advanced simulation in undergraduate pilot 
training; Automatic instructional system (AFHRL-TR-75-59(IV) , AD-A017 
165). Wright-Patterson AFB, OH: Advanced Systems Division, Air Force 
Human Resources Laboratory. 

Faconti, V., Mortin^er, C.P.L., & Simpson, D.W. (1970). Automated 

instruction and performance monitoring in flight slmu1ato~training 
IAFHRL-TR-69-Z9, Au-704 120). Wright-Patterson AFB, OH: Training 
Research Division, Air Force Human Resources Laboratory. 

Feuge, R.L., Charles, J. P., & Miller, R, (1973). Fea sibility of automated 
adaptive GCA (ground controlled approach) controller training system 
INAVTRAEQUIPCEN-73-C-0079-1, AD-778 3l2). Orlando, PL: Naval 
Training Equipment Center. 

Fuller, J.H., Waag, W.L., & Martin, E.L. (1980). A dvanced simulator for 
PlI?^. ^r?^*[!j"91 Design of automated pfe>rformance measurement system 
(AFHRL-TR-79-57, Ab-A088 855). Will lams AFB, AZ: Operations Training 
Division, Air Force Human Resources Laboratory. 

Gray, T.H., Chun, E.K., Warner, H.D., & Eubanks, J,L. (1981). Adv anced 
flight simulator: Utilization in A-10 conversion and air-to-surface 
attack training tAFHRL-TR-80-ZO, AD-A094 608). Williams AFB, AZ: 
Operations Training Division, Air Force Human Resources Laboratory. 

Hilgard, E.R., & Bower, G.H. (1975). Theories of learnin g (4th ed.). 
Englewood Cliffs, NJ: Prentice-Hall, Inc. 

Hughes, R.G. (1979). Advanced training features: Bridging the gap between 
in-flight and simulator-based models of flying training 
lAFHRL-TR-78-96, AD-A068 142). Williams AFB, AZ: Flying Training 
Division, Air Force Human Resources Laboratory. 

Hughes, R.G., Hannon, S.T., ? Jones, W.E. (1979). Application of flight 
simulator record/playback feature (AFHRL-TR-79-52, AD-A081 75Z). 
Williams AFB, AZ: Flying Training Division, Air Force Human Resources 
Laboratory. 

Hughes, R.G., Lintern, G., Wightman, D.C., Brooks, R.B., & Singleton, J. 
(1982). Applications of simulator freeze^ to carrier glldeslope 
tracking instruction (AFHRL-TR-82-3, AD-A118 862). Williams AFB, AZ: 
Operations Training Division, Air Force Human Resources Laboratory. 

K:'»oop, P. A. (1973). Advance instructional provisions and automated 
performance measurement. Human Factors , 15 , 583-597. 

Martin, E.L., & Waag, W.L. (1978a). Contributions of platform motion to 
simulator training effectiveness: study :: Basic contact 
(AFHRL-TR-78-15, AD-A058 416). Williams AFB, AZ: Flying Training 
Division, Air Force Human Resources Laboratory* 



83 

53 



Martin,, E.L., & Waag, W.L. (1978b). Contributions of platform motion to 
simulator training effectiveness: study II: Aerobatics 
(AFHRL-TR-78-62, AD-A064 305). Williams AFB, AZ: Flying Training 
Division, Air Force Human Resources Laboratory. 

Polzella, D.J. (1983). Aircrew training devices: Utility and utilization of 
advanced Instructional features (Phase I - Tactical Air Command) 
(AFHRL-TR-83-22, AD-A135 052). Will lams AFB, AZ: Operations Training 
Division, Air Force Human Resources Laboratory. 

Polzella, D.J. (1985). Aircrew training devices: Utility and utilization o f 
advanced instructional features (Phase II - Air T^^"?"?"? ,^Q^^g"j>" 
in ^itary Airlift^ Command, and Strategic Air Command} (AFHRL-TR-85-48, 
r J'A166 726. Williams AFB, AZ: Operations Training Division, Air 
Force Human Resources Laboratory. 

Polzella, D.J., & Hubbard, D.C. (1985). AiYcrew training devices: Utility 
and utilization of advanced instructional features (Phase III - 
electronic warfare trainers) (AFHRL-TR-85-49, AD-A167 922). W11 1 iams 
AFB, AZ: Operations Training Division, Air Force Human Resources 
Laboratory. 

Ricard, G.L., Parrish, R.V., Ashworth, B.R., & Wells, M.D. (1981). The 
effects of various fidelity factors on simulated helicopter hover 
(NAVTRAEQUIPCEN-IH-321, AD-A102 028). Orlando, FL: Naval Training 
Equipment C«5nter. 

Semple, C.A., Cotton, J.C., & Sullivan, D.J. (1981). ^j^^gl^^^^'^^^j" j"9 
devices: instructional support features ( AFHRL-TR-80-58, AD-A096 
234). Wright- Patterson ArB, OH: Logistics and Technical Training 
Division, Air Force Human Resources Laboratory. 

United States General Accounting Office (1983). Greater benefits to be 
gained from Do D flight simulators '.AD-A123 7W. Alexandria, VA: 
Defense Technical Information Center, Defense Logistics Agency. 



84 

54 



APPEl'^OIX A: SIMULATOR INSTRUCTORS SURVEYED IN PHASES I, II, AND III 
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Table A-1 . Simulator Instructor* (Sis) Surveyed in Phase I 

Level of Type Instructor 

Command ATP ATD-Sites training of SI N hours 

TAC F-4E George Replacement IP 16 242.2 

F-4G AFB, CA (290.6) 

George Continuation IP 10 171.5 

AFB, CA (192.0) 

F-15 Luke Replacement IP 20 171.2 

AFB, AZ (108.9) 

Eglin AFB, FL Continuation IP 19 115.3 

Langley AFB, VA (107.7) 

A-10 Davis-Monthan Replacement IP 26 21.9 

AFB, AZ (39.9) 

Myrtle Beach Continuation IP 12 21.2 

AFB, SC (18.3) 

E-3A Tinker Replacement IP 5 362.4 

Flight AFB, OK (476.4) 



Simulator 



Tinker Continuation IP 3 332.7 

AFB, OK (103.0) 



E-3A Tinker Replacement WDI 17 529.4 
Mission AFB, OK (491.8) 
Simulator 

Tinker Continuation WDI 6 217.7 
AFB, OK (293.1) 
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Command ATD 



Table A-2 » Simulator Instructors (Sis) Surveyed in Phase II 

N 



ATD-Sites 



Level of 
training 



Type 
of SI 



Instructor 
hours 



ATC T-50 Williams 

AFB, AZ 

T-51 Williams 
AFS, AZ 

MAC C-5A Altus 

AFB, OK 

Dover 
AFB, DE 

C-Hl Altus 
AFB, OK 

McGuire 
AFB, OK 

C-130 Little Rock 
AFB, AR 

Little Rock 
AFB, AR 

CH-3 Kirtland 
AFB, NM 

Kirtland 
AFB, NM 

HH-53 Kirtland 
AFB, NM 

Kirtland 
AFB, NM 

SAC FB-lllA Plattsburgh 

AFB, NY 

Plattsburgh 
AFB, NY 
Pease AFB, NH 



Basic 



Basic 



Formal 



Formal 



Formal 



Formal 



Formal 



IP 



IP 



Operational IP,IFE 



Operational IP,IFE 



IP,IFE 



Operational IP,IFE 



IP,IFE 



Operational IP,IFE 



IP,IFE 



Operational IP, IFE 



Transition ^P,IRN 



29 
21 



IP, IFE 29 



IP,IFE 40 



12 
17 



Operational IP,IRN 45 



173.6 
(169.5) 

129.4 
(96.8) 

511.1 
(452.1) 



17 454.1 
(386.8) 



582.5 
(531.4) 



13 1174.6 
(1504.1) 

21 126.8 
(80.4) 

13 419.8 
(189.9) 

6 169.2 
(94.4) 

f. 263.4 
(132.7) 

5 139.2 
(149.8) 



482.5 
(452.1) 

797.1 
(693.3) 

353.0 
(818.4) 



273" 
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Table A-3 . Simulator Instructors (Sis) Surveyed in Phase III 



LOfTliIlanu MIU 


Aiu-bi tes 


Level of 
training 


Type 
of SI 


N 


Instructor 
hours 


ATC T-5 


Mather 
AFB, CA 


Basic 


lEW 


19 


287.4 
(276.0) 


SAC T-4 {B-52) 


Castle 
AFB, CA 


Trans iti or. 


lEW 


20 


731.2 
(754.6) 




Mather 
AFB, CA 


Operational 


lEW 


8 


188.9 
(224.6) 


WST (B-5?) 


Castle 
AFB, CA 


Transition 


JEW, I AG 


9 


674.& 
(747.8) 




Wurtsmith 
AFB, MI 


Operational 


lEW.IAG 


6 


396.2 
(231.9) 


*FB-mA 


Plattsburgh 
AFB, NY 


Transition 


IRN 


11 


677.3 
(426.2) 




Plattsburgh 
AFB, NY 


Operational 


IRN 


9 


175.6 
(213.8) 




Pease 
AFB, NH 


Operational 


IRN 


12 


210.2 
(182.7) 


TAC F-4G 


George 

A CD A 

ArB, CA 


Replacement 


lEW 


13 


128.4 
(116.1) 




George 
AFB, CA 


CoHu muation 


lEW.IP 


19 


73.1 
(56.4) 


A-10 


Davis-Monthan 
AFB, AZ 


Replacement 


IP 


16 


98.8 
(85.8) 




England 
AFB, LA 


Continuation 


IP 


17 


52.6 
(37.4) 



159 



* Data from these sites were collected during Phase II. 
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APPENDIX B: SURVEYS FOR PHASES I, II, AMD III 
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PHASE I QUESTIONNAIRE 
ADVANCED INSTRUCTIONAL FEATURES - IP SURVEY 



Name 



FLYING EXPERIENCE: 
Aircraft 



Rank 



Squadron 



Date 



Total Hours 



IP Hours 



SIMULATOR EXPERIENCE: 
Simulator 



Total Hours 



IP Hours 



BRIEFLY DESCRIBE A "TYPICAL" TRAINING SESSION ON THIS SIMULATOR: 
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PHASE I QUESTIONNAIRE (continued) 



Please familiarize yourself with tNese instructional features and their 
definitions: 

Instructor Pilot Tutorial - provides the IP with self sDaced programned 
instruction in the capabilities and use of the flight simulator. 

Reset - permits instructor to "return" the simulated aircraft to a stored 
set nf conditions and parameters. 

Total System Freeze - permits instructor to interrupt and suspend 
simiaated flight by freezing all system parameters. 

Ref ^rded Briefing - permits instructor to provide student with information 
ao'jut a structured training session through audio/visual media 
' presentation. 

Demonstration - permits instructor to demonstrate aircraft maneuver (s) by 
prerecording and subseouently playing back a standardized segment of 
simulated flight. 

Record/Playback - permits instructor to record and subsequently playback 
aii events uiat occurred during a segment of simulated flight. 

Ehvironmental - permits instructor to vary environrental conditions such 
as wind direction and velocity, turbulence, temperature, visibility, etc. 

Automated Malfunction Insertion - permits instructor to pre-program a 
sequence of aircraft component malfunctions and/or emergency conditions. 

Flight System Freeze - permits instructor to simultaneously freezs flight 
control and propulsion systems, latitude, longitude, altitude, and heading. 

Position Freeze - permits instructor to simultaneously freeze latitude and 
longitude. 

Parameter Freeze - permits instructor to freeze any one or coirtoination of 
flicTit parameters. 

Crash and/or Kill Override - permits instructor to allow simulated flight 
to continue without Interruption following a "crash" or "kill." 

Motion - permits instructor to vary platform motion system cues such as 
roll, pitch, lateral, vertical, etc. 

Automated Performance Feadback - proviJes student with visual and/or 
auditory signals (including verbal messages) that identify ''performance 
detlclencies. 

Hard Copy - provides a record of alphanumeric and/or graphic performance 
oata from the automated performance measurement system for debriefing 
purposes* 

Automated Adaptive Training - computer-controlled variations in task 
difficulty, complexity, and sequence based on pilot's performance. 

Programmed Mission Scenarios - computer -con trolled standardized training 
sessions based on pre-programmed event sequences. 
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PHASE I QUESTIONNAIRE (Continued) 

U Hm onm twv« you uMd «cD tmtsueUoMl fiMturtT iOmk tht trmprUte mm.) 
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PHASE ! QUESTIONNAIRE (Continued) 
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PHASE I QUESTIONNAIRE (Continued) 

, , , Jn tht vst 9f ttch \n% 
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PHASE I QUESTIONNAIRE (Continued) 
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PHASE I QUESTIONNAIRE (Concluded) 
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TbtrtfBrf» boat your ratlngi m tha fNturt diftnltiona alona. (Oiadc the approptlata ipaoa.) 
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PHASE II QUESTIONNAIRE 

ADVANCED INSTRUCTIONAL FEATURES - IP SURVEY 



Name ^Rank ^Squadron Date 



FLYING EXPERIENCE: 

Aircraft Total Hours IP Hours 



SIMUUTOR EXPERIENCE: 

Simulator Total Hours IP Hours 



BRIEFLY DESCRIBE A "TYPICAL" TRAINING SESSION ON THIS SIMJLATOR: 



GENERAL COMMENTS AND/OR RECOMMENDATIONS: 
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PHASE II Questionnaire (Continued) 

Please familiarize yourself with these instructional features and their 
definitions: For eacn feature, insert 1 ((* ^ ^ailable) or 0 ^unavailable; : 

DM 

Instructor Pilot Tutorial - provides the IP with self-paced prograrrmed 

instruction in the capabilities and use of the flight simulator. 

Reset - permits instructor to "return" the simulated aircraft to a 

stored set of conditions and parameters. 

Total System Freeze - permits instructor to int '^rupt and suspend 

simulated flight by freezing all system parameters, 

Recorded Briefing - permits instructor to provide student with 

information about a structured training session through audio/visual 
media presentation. 

Demonstration - permits instructor to demonstrate aircraft maneuver Is) 

by prerecording and subsequently playing back a standardized segment 
of simulated flight. 

Record/Playback - permits instructor to record and subsequently 

playback all events that occurred during a segment of simulated flight. 

Environmental - permits instructor to vary environmental conditions 

such as wind direction and velocity, turbulence, temperature, 
visibility, etc. 

Automated Malfunction Insertion - permits instructor to pre-program a 

sequence of aircraft component malfunctions and/or emergency 
conditions. 

Partial Freeze - permits instructor to freeze various flight 

parameters or parameter combinations such as altitude, heading, 
position, attitude, flight system, etc. 

Crash and/or Kill Override - permits instructor to allow simulated 

flight to continue without interruption following a "crash" or "kill." 

Motion - permits instructor to vary platform motion system cues such 

as roll, pitch, lateral, vertical, etc. 

Hard Copy orovldes a record of alphanumeric and/or graphic 

performance u^.^ from the automated performance measurement system for 
debriefing purpose 

Automated Adaptive Tra.. - computer-controlled variations in task 

difficulty, complexity, and sequence based on pilot's performance. 

Programned Mission Scenarios - computer-controlled standardized 

training sessions based on pre-prograrmied event seqi-f^nces. 

Procedures Monitoring - permits instructor to monitor discrete actions 

performed by the student in accordance with a procedurally defined 
checklist. 

Parameters Monitoring - permits instructor to monitor various 

instrument readings, control settings, aircraft states, or 
navigational profiles. 
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PHASE II QUESTIONNAIRE (Continued) . 

t* How Often hm you used e«ch Instructional feature? (Check the appropriate space.) 

/moderately/ / very / 

Fejturj /never/rartly/occaslonally/ often /frequently/frequently/most often 
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PHASE II QUESTIONNAIRE (Continued) 

2. How easy is it to use each instruct foni I feature? (Check the ipproprlite spice.) 

/ most / very / 

f»*ture / 7 /dlfficuU/dlffkuIt/difficuIt/noderite/ easy /very easy/easiest/ 
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PHASE 11 QUESTIONNAIRE (Continued) 

3. Nm mxn triinlng did 2SI rectUt in tnt use of e«ch instruction* 1 feiture? {Chtct tne appropriate space. 
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, , ' , PHASE II QUESTIONNAIRE (Continued) 

4, Rate the triining vilue of eich 1n$tructionil feature. (Check the appropriite jpice.) 
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PHASE II QUESTIONNAIRE (Concluded) 



S DotwtliJ trilRlni vtJui of ttcii InitructlontJ fwiurt, includtnf tjost you *rt npt fuHUr mtn. 

Kiirth?rrtyK4»^ hi! no tiMrHftCt utln« «ny of tht ftrturtt tiMl thU tU of thM «ri cqutli/ t«y to w 
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PHASE III QUESTIONNAIRE 
ADVANCED INSTHJCTICNAL FEATURES - EWI SURVEY 



Name 



FLnNG EXPERIEUCE: 
Aircraft 

SIMJLATOR EXPERIENCE: 
Simulator 



Rank 



Squadron 



Total Hours 



Total Hours 



Date 



Instructor Hours 



Instructor Hours 



L What percent of your initial instruction on simulation training consisted of formal 
classroom instruction and what percent consisted of informal instruction? 



% formal classroom 



% informal 



2. Have you had refresher training on simulation operation? 

(If no, skip next two items.) 

a. How long has it been since yoo last had refresher training? 



yes 



no 



weeks 



b. What percent of your refresher training was formal and what percent informal? 
% formal classroom % informal 



BRIEELY DESCRIBE A "TYPICAL" TRAINING SESSION CN THIS SIMJLfiTOR: 



GENERAL COtENTS AM5/0R REOOH^ElDAriCtB : 



PHASE III QUESTIONNAIRE (Continued) 



Read the definitions of each inr>tructional feature carefully. In the 
space next to each feature, write the single number corresponding to the 
statement tha^t best describes the operational status of that feature: 

0. The simulator has no such capability. 

!• Capability present but I have never seen it operate. 

2. Capability present but unreliable. 

3. Capability present and reliable. 



Instructor Tutorial - provides the instructor with self-paced 
programmed instruction in the capabilities and use of the simulator. 

Res^t - permits instructor to '•return'' the simulated aircraft to a 
stored set of conditions and parameters. 

Total Syst em Freeze - permits instructor to interrupt and suspend 
simulated flight by freezing all system parameters. 

Partial -Freeze • permits instructor to freeze various flight 
pararoeters or p a r am e te r combinations such as altitude, heading, 
position, attitude, flight system, etc« 

Recorded Brief inq - permits instructor to provide student with 
information about a structured training session through audio/visual 
media presentation. 

Demonstration - permits instructor to demonstrate optimal electronic 
warfare procedures by prerecording and subsequently playing back a 
simulated engagement. 

Reoord/ I Playback - permits instructor to record and subsequently 
playback a segment of simulated flight. 

Automated Malfunction Insertion - permits instructor to pre-prograra a 
sequence of aircraft conponent malfunctions and/or emergency 
conditions. 

Hard Copy - provides a record of alphanumeric and/or graphic 
performance data for debriefing purposes. 

programmed Threat Control - conputer-controlled standardized training 
sessions based on pre-programmed event sequences. 

Manual Threat Control - permits instructor to modify threat scenarios 
during a training session. 

Procedures Monitoring - permits instructor to monitor discrete actions 
performed by the student in accordance with a procedurally defined 
checklist. 

Parameters Monitorincr - permits instructor to monitor various 
instmnent readings, control settings, aircraft states, or 
navigational profiles. 

Electronic Warfare Performance Scoring - provides a performance metric 
that sumarizes the outcomes of BH[ engagements*^ 
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PHASE III QUESTIONNAIRE (Continued) 

I. Ourinj fiv^ typicAi sdwiont, how ofun did yog um tach initcuctiontX fwturt? {C^ trw jf^propriau spact.) 

/ / / Onct Evtcy 5 / Onct Evtcy / « / 2-4 Tla»« / S.-; t4«««/ a ^ ^ 

/ S/A / s.m / «imon. or / 2-4 tMt^^ , juSioJ /. //'«t JiS^^Tlm.V^xmt, / 

Initruetoc Tutociil / / / / / / / * 

4 1 2 3 ^-4 ^— I ^ , f 

Comantst 

5^ T"-^ 3 J 4 '—I s ^ J ^ 

OOMMieSI 

Tout syttw rf«n / / / / / / / / 

^^^^^^ 1 ^ — ^ ^ ' , ^ , ^ , - , ^ 

CoMnut 



Cammtss 



Racord/PliiyDack / / / / / / / / 



Qammxsx 



CoomnUt 
Proqrinwd Thr—t 



^ ^ a ^ 1 ^ ^ ^ J ^ , ^ , ^ , 

Maftoal Thfit Cofitfol / / / / / / / 

i ' I ^ J ^ 3 ^ * ^ s ^— r 



Proctto^ Kpftltoriryi / / / ^ / / / / / / 

0 > 2 3 — 5 1 J — ^ 

P.ri.uc.moltotln,^ a ^ ^ ^ J ^ J ^ ^ / ^ / ^ / ^ / 

OommtMi 
ei>ctfo>iie wtrf ig» 

2 J 4 — — S ^ J ^ 

CammtMt 
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PHASE III QUESTIONNAIRE , Continued) 

2. How dieeicult/tMy la it to ttB« Mch inttcuctio.-Ml f««tu£«? {Qmn m sppropcxiu spftc«.l 

/ / VlctuaUy / Vtcy / Ftlcly / N«ltr.«c E*»y / faicly / v«o' / vutatUy / 
/ H/A / If|»Mlbl« / Difficult / OUflcult / Nor Difficult / Ea«y / EAty / Autorwt^c / 

Intttuctoc TutcclU / / / / / / / / / 

0 i * J * 5 5 7 



OoaHntat 



Toul sntm rrww / / .., i . i , L / i / / 

0 i 2 3 4 5 6 7 



Mcertrt ■rt.flw / / ^ i C i / L—Z , / 

15 1 2^ J 4 5 S 7 



0 I 2 3 4 5 5 7 



Uoot^UNtmk I I i L , L L—L f 

^ I 2 7^ ^ 5 5 7 

CiiMirf t 
Auttanatta Mfttf unction 

J-—, I 2 5 4 5 6 7 
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PcqgCMud Thf— t 

/ / / _/ ,/ ^ . ^ ^ ^ ^ 

^ a I 2 3 i 5 S 7 



Manual TTtf—t Control / / , ^ . ^ . ^ . ^ 

0 1 2 3 4 5 o 7 



Procatoaattenit orlnti/ ^ / ^ ^ ^ i ^ L ^ / ^ / ^ f 



Hraaataca Hanitprtmi / / / / / / / / _ / 

1 5 3 I 5 5 ? 



Blaetronic Wacfara 

*1^<<'Tf^*9»'W / ^ / J ^ J L 3 (• , , f 
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PHASE III QUESTIONNAIRE (Continued) 



3« How inadequau/adequau was tht training you ceceivad in the use of each instrucuonai feature? 
(ChacJc ttMi acpropriau spaot*) 

/ No / Totally / Very / SUghtiy / / SUghUy / Very / Totjiiw / 
/ Tcaininq / Inadequate / Inadeouate / Inadeou^te / RArH«ri(n* / tA^,>^^ / ha / 
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PHASE III QUESTIONNAIRE (Continued) 



As presently ittplwnted on your systeair ho** useful is each instructional feature? (Oiedc the appropriate space 

/ / Not / Slightly/ Fairly / Moderately/ Very / Extrer^ly / / 
/ N/A / useful / Useful / Useful / Useful / Useful / Useful / Indispensable / 
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PHASE III QUESTIONNAIRE (Concluded) 

5« Bumi on tht atflnlUont aim and not youc exptdtnot, how potentiaiy uMfuX U Mch Iratructiooal f Mtur* . 
faatuct* ASM Mch U •quaUy auy to um. (QmcIc thi appropciau spin.) iMoict* Ratt < 
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